In a Django template, how to specify a dictionary

Question

I'm working on a Django project with a ListView that has both a search form, known in the view's context as search_form, and a filter form, filter_form. The interface looks like this:

Both the search_form and the filter_form ultimately change what is returned by the ListView's get_queryset method. I would like to make it such that when you first apply a filter and then do a search, it will search the filtered results.

At the moment, the 'reverse' functionality has already been implemented: when you first search and then filter, it will filter the search results. This is because of a hidden input element in the filter_form:

<form action={% url 'dashboard:families' %} method="GET" data-behavior="filters">
                <input type="hidden" name="q" value="{{ request.GET.q.strip }}"/>

                <div class="input-field col s2">
                  {{ filter_form.guide }}
                  <label class="active">Guide</label>
                  {% if filter_form.is_guide_filled %}
                    <a href="" class="clear"><i class="material-icons tiny">clear</i></a>
                  {% endif %}
                </div>

For comparison, the search bar has the following template, _search.html:

<form action="{% url action %}" method="get" class="left search col s6 hide-on-small-and-down" novalidate>
  <div class="input-field">
    <input id="search" placeholder="{{ placeholder }}"
        autocomplete="off" type="search" name="q"
        value="{{ search_form.q.value.strip|default:'' }}"
        data-query="{{ search_form.q.value.strip|default:'' }}">
    <label for="search" class="active"><i class="material-icons search-icon">search</i></label>
    <i data-behavior="search-clear"
        class="material-icons search-icon"
        {% if not search_form.q.value %}style="display: none;"{% endif %}>close</i>
  </div>
</form>

In the main list view, index.html, the search template is included like so:

{% block search_form %}
  {% with action='dashboard:families' placeholder='Search Families' %}
    {% include '_search.html' %}
  {% endwith %}
{% endblock %}

In order to make the search form persist the filters, I've noticed that for the guide field only, the following works:

<input type="hidden" name="guide" value="{{ request.GET.guide }}"/>

I would like to generalize this to include all the filters. I've tried the following:

  {% if filter_form %}
    {% for field in filter_form %}
      {% with field_name=field.name %}
        <input type="hidden" name=field_name value="{{ request.GET.field_name}}"/>
      {% endwith %}
    {% endfor %}
  {% endif %}

However, if I try this I literally get "field_name" in the querystring:

As I understand from the DTL docs, the dot-notation implements dictionary lookup, attribute lookup, and list-index lookup. If I were to try something like

request.GET.field.name

it would probably try to look up "field" in the request.GET dictionary-like object and not find anything. In regular Python, what I'd essentially like to do is

request.GET[field.name]

I thought I could make this work with a with block, but apparently this doesn't work. Any advice on how to implement this?

Update

If I specify the name attribute for the input element as "{{field_name}}" instead of just field_name, like so,

<input type="hidden" name="{{field_name}}" value="{{ request.GET.field_name}}"/>

The problem is that the value is being set to an empty string instead of the desired value, which leads to a ValueError for the guide field (which is a ModelChoiceField expecting an integer as input):

Why is the attribute lookup not working in this case?

Update 2

Responding to Lemayzeur's comment, the full traceback is:

Traceback:

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
  41.             response = get_response(request)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/views/generic/base.py" in view
  68.             return self.dispatch(request, *args, **kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/two_factor/views/mixins.py" in dispatch
  82.         return super(OTPRequiredMixin, self).dispatch(request, *args, **kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/contrib/auth/mixins.py" in dispatch
  56.         return super(LoginRequiredMixin, self).dispatch(request, *args, **kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/contrib/auth/mixins.py" in dispatch
  92.         return super(PermissionRequiredMixin, self).dispatch(request, *args, **kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/views/generic/base.py" in dispatch
  88.         return handler(request, *args, **kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/dashboard/views/families.py" in get
  74.         return super().get(request, *args, **kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/dashboard/views/base.py" in get
  111.         return super().get(request, *args, **kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/dashboard/views/base.py" in get
  74.         return super().get(request, *args, **kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/views/generic/list.py" in get
  160.         self.object_list = self.get_queryset()

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/dashboard/views/families.py" in get_queryset
  122.             queryset = queryset.filter(lucy_guide__in=guide)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/db/models/query.py" in filter
  784.         return self._filter_or_exclude(False, *args, **kwargs)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/db/models/query.py" in _filter_or_exclude
  802.             clone.query.add_q(Q(*args, **kwargs))

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/db/models/sql/query.py" in add_q
  1250.         clause, _ = self._add_q(q_object, self.used_aliases)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/db/models/sql/query.py" in _add_q
  1276.                     allow_joins=allow_joins, split_subq=split_subq,

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/db/models/sql/query.py" in build_filter
  1206.             condition = lookup_class(lhs, value)

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/db/models/lookups.py" in __init__
  24.         self.rhs = self.get_prep_lookup()

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/db/models/fields/related_lookups.py" in get_prep_lookup
  56.                 self.rhs = [target_field.get_prep_value(v) for v in self.rhs]

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/db/models/fields/related_lookups.py" in <listcomp>
  56.                 self.rhs = [target_field.get_prep_value(v) for v in self.rhs]

File "/Users/kurtpeek/Documents/Dev/lucy/lucy-web/venv/lib/python3.6/site-packages/django/db/models/fields/__init__.py" in get_prep_value
  966.         return int(value)

Exception Type: ValueError at /dashboard/families
Exception Value: invalid literal for int() with base 10: ''

If I set a trace in the view's get_queryset() method, just before 122 in the stack trace, I see that guide is indeed an empty list containing an empty string, as it is in self.request.GET:

> /Users/kurtpeek/Documents/Dev/lucy/lucy-web/dashboard/views/families.py(121)get_queryset()
    120 
--> 121         if guide:
    122             queryset = queryset.filter(lucy_guide__in=guide)

ipdb> guide
['']

ipdb> self.request.GET
<QueryDict: {'q': ['Christine'], 'status': [''], 'next_outreach': [''], 'country': [''], 'vip': [''], 'app': [''], 'guide': [''], 'package': ['']}>

Aside from the difficulties in submitting the empty value, I actually would expect the guide field of request.GET to have a non-empty value here. For example, if I return to the 'simplified' version of the hidden filter form, with only the guide field:

<form action="{% url action %}" method="get" class="left search col s6 hide-on-small-and-down" novalidate>
  <div class="input-field">
    <input id="search" placeholder="{{ placeholder }}"
        autocomplete="off" type="search" name="q"
        value="{{ search_form.q.value.strip|default:'' }}"
        data-query="{{ search_form.q.value.strip|default:'' }}">
    <label for="search" class="active"><i class="material-icons search-icon">search</i></label>
    <i data-behavior="search-clear"
        class="material-icons search-icon"
        {% if not search_form.q.value %}style="display: none;"{% endif %}>close</i>
  </div>
  <input type="hidden" name="guide" value="{{ request.GET.guide }}"/>
</form>

Then I select a filter, and enter a search term in the search bar:

Then when I set a trace just after the view's get method, like so:

def get(self, request, *args, **kwargs):
    import ipdb; ipdb.set_trace()

I do see that both q and guide are in request.GET:

ipdb> request.GET
<QueryDict: {'q': ['Christine'], 'guide': ['6']}>

So in the 'general' form with request.GET.field_name, I would also expect the request.GET to look like this, with also empty lists for the other fields. It seems like the Django Template Language's dot notation is trying to literally do a dictionary or attribute lookup for 'field_name' and not finding anything; perhaps I should write a custom filter as described in Django template how to look up a dictionary value with a variable to perform the dictionary lookup of field.name?

Answer 1

I finally solved this by writing a custom get filter as described in Django template how to look up a dictionary value with a variable:

from django import template

register = template.Library()


@register.filter
def get(dictionary, key):
    return dictionary.get(key)

I updated _search.html as follows:

{% load get %}

<form action="{% url action %}" method="get" class="left search col s6 hide-on-small-and-down" novalidate>
  <div class="input-field">
    <input id="search" placeholder="{{ placeholder }}"
        autocomplete="off" type="search" name="q"
        value="{{ search_form.q.value.strip|default:'' }}"
        data-query="{{ search_form.q.value.strip|default:'' }}">
    <label for="search" class="active"><i class="material-icons search-icon">search</i></label>
    <i data-behavior="search-clear"
        class="material-icons search-icon"
        {% if not search_form.q.value %}style="display: none;"{% endif %}>close</i>
  </div>
  {% if filter_form %}
    {% for field in filter_form %}
      <input type="hidden" name="{{ field.name }}" value="{{ request.GET|get:field.name }}"/>
    {% endfor %}
  {% endif %}
</form>

Now, if I try to search a filtered result, it works as expected:

Note that this also works fine for the filters that are not applied - these have the value None instead of an empty string - without any need to filter them out in the form.

Answer 2

So it basically boils down to not submitting keys in the GET request that have an empty string value. This appears to be unsupported natively in HTML; you need some JS magic to make this happen. See this thread: How to prevent submitting the HTML form's input field value if it empty

However, a pure Django solution would be to modify your filter dict to exclude keys that are null. I am not sure how you are filtering this in Django, but assuming you have override the get_queryset method; you can always do:

def get_queryset(self):
    qs = super(YourView, self).get_queryset()
    filters = {k, v for k, v in request.GET.items() if v != ''}  # Be as generic/specific as needed here for exclusion
    qs = qs.filter(**filters)  # Fire your filtering logic here; this is a sample
    return qs