Two Scoops of Django 3.x by Daniel Audrey Feldroy (251-350)

17.
2: Illustrating Design Concepts With a Simple API
TIP: Classy Django REST Framework Is a Useful Reference

For working with the Django Rest Framework, we’ve found that http://cdrf.co
is a great cheat sheet. It is patterned after the famous ccbv.co.uk reference site,
but tailored for Django REST Framework.
Now we’ll wire this into our flavors/urls.py module:
Example 17.5: Wiring in API Views
# flavors/urls.py
from django.urls import path
from flavors.api import views
urlpatterns = [
# /flavors/api/
path(
route='api/',
view=views.FlavorListCreateAPIView.as_view(),
name='flavor_rest_api'
),
# /flavors/api/:uuid/
path(
route='api/<uuid:uuid>/',
view=views.FlavorRetrieveUpdateDestroyAPIView.as_view(),
name='flavor_rest_api'
)
]
What we are doing is reusing the URLConf name, making it easier to manage when you
have a need for a JavaScript-heavy front-end. All you need to do is access the Flavor resource
via the {% url %} template tag.
In case it’s not clear exactly what our URLConf is doing, let’s review it with a table:
Url View Url Name (same)

/flavors/api/ FlavorListCreateAPIView flavor_rest_api
/flavors/api/:uuid/ FlavorRetrieveUpdateDestroyAPIView
flavor_rest_api
Table 17.3: URLConf for the Flavor REST APIs
Please submit issues to github.com/feldroy/twoscoops- of- django- 3.x/issues 213

Prepared exclusively for LAKLAK LAKLAK ([email protected]) Transaction: 11322
Chapter 17: Building REST APIs With Django REST Framework
WARNING: Our Simple API Does Not Use Permissions

We overrode the default IsAdmin permission with IsAuthenticated. If you im-
plement an API using our example, don’t forget to assign user permissions appro-
priately!
ä django-rest-framework.org/api-guide/authentication
ä django-rest-framework.org/api-guide/permissions
The end result is the traditional REST-style API definition:
Example 17.6: Traditional REST-style API definition
flavors/api/
flavors/api/:uuid/
TIP: Common Syntax for Describing REST APIs

It’s not uncommon to see syntax like what is described in the Wiring in API Views
code example. In this particular case, /flavors/api/:uuid/ includes a :uuid
value. This represents a variable, but in a manner suited for documentation across
frameworks and languages, and you’ll see it used in many third-party REST API
descriptions.
We’ve shown you (if you didn’t know already) how it’s very easy to build REST APIs in
Django. Now let’s go over some advice on maintaining and extending them.
17.3 REST API Architecture

Building simple, quick APIs is easy with Django REST Framework, but extending and
maintaining it to match your project’s needs takes a bit more thought. This is usually where
people get hung up on API design. Here are some tips for improving your design:
17.3.1 Use Consistent API Module Naming

Just like anything else, how things are named needs to be consistent across a project. Our
preferences for naming module related to API design is as follows:
Example 17.7: Our Preferences for Naming API-Related Modules
flavors/
├── api/
214 Please submit issues to github.com/feldroy/twoscoops- of- django- 3.x/issues

17.3: REST API Architecture
│ ├── __init__.py

│ ├── authentication.py
│ ├── parsers.py
│ ├── permissions.py
│ ├── renderers.py
│ ├── serializers.py
│ ├── validators.py
│ ├── views.py
│ ├── viewsets.py
Please observe the following:
ä We like to place all our API components into a package within an app called api/ .
That allows us to isolate our API components in a consistent location. If we were to
put it in the root of our app, then we would end up with a huge list of API-specific
modules in the general area of the app.
ä Viewsets belong in their own module.
ä We always place routers in urls.py. Either at the app or project level, routers belong
in urls.py.
17.3.2 Code for a Project Should Be Neatly Organized

For projects with a lot of small, interconnecting apps, it can be hard to hunt down where
a particular API view lives. In contrast to placing all API code within each relevant app,
sometimes it makes more sense to build an app specifically for the API. This is where all
the serializers, renderers, and views are placed.
Therefore, the name of the app should reflect its API version (see Section 17.3.7: Version
Your API).
For example, we might place all our views, serializers, and other API components in an app
titled apiv4.
The downside is the possibility for the API app to become too large and disconnected from
the apps that power it. Hence we consider an alternative in the next subsection.
17.3.3 Code for an App Should Remain in the App

When it comes down to it, REST APIs are just views. For simpler, smaller projects, REST
API views should go into views.py or viewsets.py modules and follow the same guidelines we

endorse when it comes to any other view. The same goes for app- or model-specific serializers
and renderers. If we do have app-specific serializers or renderers, the same applies.
For apps with so many REST API view classes that it makes it hard to navigate a single
api/views.py or api/viewsets.py module, we can break them up. Specifically, we move our
view (or viewset) classes into a api/views/ (or api/viewsets/ ) package containing Python
modules typically named after our models. So you might see:
flavors/
├── api/
│ ├── __init__.py
│ ├── ... other modules here
│ ├── views
│ │ ├── __init__.py
│ │ ├── flavor.py
│ │ ├── ingredient.py
The downside with this approach is that if there are too many small, interconnecting apps,
it can be hard to keep track of the myriad of places API components are placed. Hence we
considered another approach in the previous subsection.
17.3.4 Try to Keep Business Logic Out of API Views
Regardless of which architectural approach you take, it’s a good idea to try to keep as much
logic as possible out of API views. If this sounds familiar, it should. We covered this in
Section 8.5: Try to Keep Business Logic Out of Views. Remember, at the end of the day,
API views are just another type of view.

Figure 17.1: An Ice Cream as a Service API.
17.3.5 Grouping API URLs

If you have REST API views in multiple Django apps, how do you build a project-wide
API that looks like this?
Example 17.9: Project-Wide API Design
api/flavors/ # GET, POST

api/flavors/:uuid/ # GET, PUT, DELETE
api/users/ # GET, POST
api/users/:uuid/ # GET, PUT, DELETE
In the past, we placed all API view code into a dedicated Django app called api or apiv1,
with custom logic in some of the REST views, serializers, and more. In theory it’s a pretty
good approach, but in practice it means we have logic for a particular app in more than just
one location.
Our current approach is to lean on URL configuration. When building a project-wide API
we write the REST views in the api/views.py or api/viewsets.py modules, wire them into

a URLConf called something like core/api_urls.py or core/apiv1_urls.py, and include that

from the project root’s urls.py module. This means that we might have something like the
following code:
Example 17.10: Combining Multiple App API Views Into One
# core/api_urls.py
"""Called from the project root's urls.py URLConf thus:
path('api/', include('core.api_urls', namespace='api')),
"""
from django.urls import path
from flavors.api import views as flavor_views

from users.api import views as user_views
urlpatterns = [
# {% url 'api:flavors' %}
path(
route='flavors/',
view=flavor_views.FlavorCreateReadView.as_view(),
name='flavors'
),
# {% url 'api:flavors' flavor.uuid %}
path(
route='flavors/<uuid:uuid>/',
view=flavor_views.FlavorReadUpdateDeleteView.as_view(),
name='flavors'
),
# {% url 'api:users' %}
path(
route='users/',
view=user_views.UserCreateReadView.as_view(),
name='users'
),
# {% url 'api:users' user.uuid %}
path(
route='users/<uuid:uuid>/',
view=user_views.UserReadUpdateDeleteView.as_view(),
name='users'
),
]

17.3.6 Test Your API

We find that Django’s test suite makes it really easy to test API implementations. It’s cer-
tainly much easier than staring at curl results! Testing is covered at length in Chapter 24:
Testing Stinks and Is a Waste of Money!. We even include in that chapter the tests we wrote
for our simple JSON API (see Section 24.3.1: Each Test Method Tests One Thing).
17.3.7 Version Your API

It’s a good practice to abbreviate the urls of your API with the version number e.g.
/api/v1/flavors or /api/v1/users and then as the API changes, /api/v2/flavors
or /api/v2/users. We prefer this method or the host name style of versioning
v1.icecreamlandia.com/api/users. When the version number changes, existing cus-
tomers can continue to use the previous version without unknowingly breaking their calls
to the API.
Also, in order to avoid angering API consumers, it’s critical to maintain both the existing
API and the predecessor API during and after upgrades. It’s not uncommon for the depre-
cated API to remain in use for several months.
When we implement a new version of the API, we provide customers/users with a dep-
recation warning well in advance of breaking API changes so they can perform necessary
upgrades and not break their own applications. From personal experience, the ability to send
a deprecation warning to end users is an excellent reason to request email addresses from
users of even free and open source API services.
Django REST Framework has built-in mechanisms for supporting the preferred
scheme we list above as well as other approaches. Documentation for this exists at
django-rest-framework.org/api-guide/versioning/
17.3.8 Be Careful With Customized Authentication Schemes

If you’re building an API and need a customized authentication scheme, be extra careful.
Security is hard, and there are always unpredictable edge cases, which is how people pen-
etrate sites. We’ve only had to implement customized authentication schemes a few times,
but we kept the following in mind:
ä If we’re creating a new authentication scheme, we keep it simple and well tested.
ä Outside of the code, we document why existing standard authentication schemes are
insufficient. See the tipbox below.
ä Also outside of the code, we document in depth how our authentication scheme is
designed to work. See the tipbox below.

ä Unless we are writing a non-cookie based scheme, we don’t disable CSRF.
TIP: Documentation Is Critical for Customized Authentication

Writing out the why and how of a customized authentication scheme is a critical
part of the process. Don’t skip it! Here’s why:
ä Helps us validate our reasoning for coming up with something new. If we
can’t describe the problem in writing, then we don’t fully understand it.
ä Documentation forces us to architect the solution before we code it.
ä After the system is in place, later the documentation allows us (or others) to
understand why we made particular design decisions.
17.4 When DRF Gets in the Way

Django Rest Framework is a powerful tool that comes with a lot of abstractions. Trying to
work through these abstractions can prove to be extremely frustrating. Let’s take a look on
overcoming them.
17.4.1 Remote Procedure Calls vs REST APIs

The resource model used by REST frameworks to expose data is very powerful, but it doesn’t
cover every case. Specifically, resources don’t always match the reality of application design.
For example, it is easy to represent syrup and a sundae as two resources, but what about the
action of pouring syrup? Using this analogy, we change the state of the sundae and decrease
the syrup inventory by one. While we could have the API user change things individually,
that can generate issues with database integrity. Therefore in some cases it can be a good idea to
present a method like sundae.pour_syrup(syrup) to the client as part of the RESTful
API.
In computer science terms, sundae.pour_syrup(syrup) could be classified as a Remote

Procedure Call or RPC.
References:
ä en.wikipedia.org/wiki/Remote_Procedure_Call
ä en.wikipedia.org/wiki/Resource-oriented_architecture
Fortunately, RPC calls are easy to implement with Django Rest Framework. All we have to
do is ignore the abstraction tools of DRF and rely instead on its base APIView:

17.4: When DRF Gets in the Way
Example 17.11: Implementing pour_syrup() RPC with DRF
# sundaes/api/views.py
from django.shortcuts import get_object_or_404
from rest_framework.response import Response

from rest_framework.views import APIView
from ..models import Sundae, Syrup

from .serializers import SundaeSerializer, SyrupSerializer
class PourSyrupOnSundaeView(APIView):
"""View dedicated to adding syrup to sundaes"""
def post(self, request, *args, **kwargs):

# Process pouring of syrup here,
# Limit each type of syrup to just one pour
# Max pours is 3 per sundae
sundae = get_object_or_404(Sundae,
,→ uuid=request.data['uuid'])
try:
sundae.add_syrup(request.data['syrup'])
except Sundae.TooManySyrups:
msg = "Sundae already maxed out for syrups"
return Response({'message': msg}, status_code=400)
except Syrup.DoesNotExist
msg = "{} does not
,→ exist".format(request.data['syrup'])
return Response({'message': msg}, status_code=404)
return Response(SundaeSerializer(sundae).data)
def get(self, request, *args, **kwargs)

# Get list of syrups already poured onto the sundae
sundae = get_object_or_404(Sundae,
,→ uuid=request.data['uuid'])
syrups = [SyrupSerializer(x).data for x in
,→ sundae.syrup_set.all()]
return Response(syrups)
And our API design would look like this now:

Example 17.12: Sundae and Syrup API Design
/sundae/ # GET, POST

/sundae/:uuid/ # PUT, DELETE
/sundae/:uuid/syrup/ # GET, POST
/syrup/ # GET, POST
/syrup/:uuid/ # PUT, DELETE
17.4.2 Problems With Complex Data

Okay, we’ll admit it, we make this mistake with DRF about once a month. Let’s sum up
what happens in very simple terms with the following API design:
Example 17.13: A Cone and Scoop API
/api/cones/ # GET, POST

/api/cones/:uuid/ # PUT, DELETE
/api/scoops/ # GET, POST
/api/scoops/:uuid/ # PUT, DELETE
1 We have a model (Scoop) that we want represented within another (Cone)

2 We can easily write a GET of the Cone that includes a list of its Scoops
3 On the other hand, writing a POST or PUT of Cones that also adds or updates
its Scoops at the same time can be challenging, especially if it requires any kind of
validation or post processing
4 Frustration sets in and we leave to get some real-world ice cream
While there are nicely complex solutions for nested data, we’ve found a better solution. And
that is to simplify things just a bit. Example:
ä Keep the GET representation of the Cone that includes its Scoops
ä Remove any capability of the POST or PUT for the Cone model to modify Scoops
for that cone.
ä Create GET/POST/PUT API views for Scoops that belong to a Cone.
Our end API will now look like this:

17.4: When DRF Gets in the Way
Example 17.14: A Cone and Scoop API
/api/cones/ # GET, POST

/api/cones/:uuid/ # PUT, DELETE
/api/cones/:uuid/scoops/ # GET, POST
/api/cones/:uuid/scoops/:uuid/ # PUT, DELETE
/api/scoops/ # GET, POST
/api/scoops/:uuid/ # PUT, DELETE
Yes, this approach does add extra views and additional API calls. On the other hand, this
kind of data modeling can result in simplification of your API. That simplification will result
in easier testing, hence a more robust API.
For what it’s worth, if you take a close look at the Stripe API reference (stripe.com/
docs/api) you’ll see they follow our pattern. You can view complex data, but you have to
create it bit-by-bit.
17.4.3 Simplify! Go Atomic!

In the previous two subsections (RPC Calls and Problems With Complex Data), we’ve
established a pattern of simplification. In essence, when we run into problems with DRF
we ask these questions:
ä Can we simplify our views? Does switching to APIView resolve the problem?
ä Can we simplify our REST data model as described by views? Does adding more
views (of a straightforward nature) resolve the problem?
ä If a serializer is troublesome and is outrageously complex, why not break it up into
two different serializers for the same model?
As you can see, to overcome problems with DRF, we break our API down into smaller,
more atomic components. We’ve found that it’s better to have more views designed to be
as atomic as possible than a few views with many options. As any experienced programmer
knows, more options means more edge cases.
Atomic-style components help in these regards:
ä Documentation is easier/faster because each component does less

ä Easier testing since there are less code branches
ä Bottlenecks are easier to resolve because chokepoints are more isolated
ä Security is better since we can easily change access per view rather than within the
code of a view

17.5 Shutting Down an External API

When it’s time to shut down an older version of an external API in favor of a new one, here
are useful steps to follow:
17.5.1 Step #1: Notify Users of Pending Shut Down

Provide as much advance notice as possible. Preferably six months, but as short as one month.
Inform API users via email, blogs, and social media. We like to report the shutdown noti-
fication to the point that we worry people are getting tired of the message.
17.5.2 Step #2: Replace API With 410 Error View

When the API is finally shut down, we provide a simple 410 Error View. We include a very
simple message that includes the following information:
ä A link to the new API’s endpoint.

ä A link to the new API’s documentation.
ä A link to an article describing the details of the shut down.
Below is a sample shutdown view that works against any HTTP method:
Example 17.15: Code for a Shutdown
# core/apiv1_shutdown.py
from django.http import HttpResponseGone
apiv1_gone_msg = """APIv1 was removed on April 2, 2017. Please

,→ switch to APIv2:
<ul>
<li>
<a href="https://www.example.com/api/v3/">APIv3
,→ Endpoint</a>
</li>
<li>
<a href="https://example.com/apiv3_docs/">APIv3
,→ Documentation</a>
</li>
<li>
<a href="http://example.com/apiv1_shutdown/">APIv1 shut
,→ down notice</a>
</li>
</ul>

17.6: Rate-Limiting Your API
"""
def apiv1_gone(request):
return HttpResponseGone(apiv1_gone_msg)
17.6 Rate-Limiting Your API

Rate limiting is when an API restricts how many requests can be made by a user of the API
within a period of time. This is done for a number of reasons, which we’ll explain below.
17.6.1 Unfettered API Access Is Dangerous

In the ancient times (2010) we launched the djangopackages.org website. The project,
started during the Django Dash contest, was an instant hit for the Django community. We
sprinted on it constantly, and its feature set grew rapidly. Unfortunately, we quickly hit the
rate limit of GitHub’s first API. This meant that after a certain amount of API requests per
hour, we weren’t allowed to make any more until a new hour passed.
Fortunately at DjangoCon 2010 we had the opportunity to ask one of the founders of
GitHub if we could have unlimited access to their API. He graciously said ‘yes’ and within
a day we could get data from GitHub as much as we wanted.
We were delighted. Our users were delighted. Usage of the site increased, people were thirsty
for data as to what were the most active projects. So desirous of data were we that every hour
we requested the latest data from GitHub. And that caused a problem for GitHub.
You see, this was 2010 and GitHub was not the giant, powerful company it is today. At 17
minutes past each hour, Django Packages would send thousands of requests to the GitHub
API in a very short period. With unfettered access we were causing them problems.
Eventually, GitHub contacted us and requested that we scale back how much we were using
their API. We would still have unlimited access, just needed to give them breathing room.
We complied, checking data once per day instead of by the hour, and at a more reasonable
rate. We continue to do so to this day.
While modern GitHub can certainly handle much, much larger volumes of API access then
it could in late 2010, we like to think we learned a shared lesson about unfettered access to
an API: Grant such access cautiously.

17.6.2 REST Frameworks Must Come With Rate Limiting

Controlling the volume of REST API access can mean the difference between joyful tri-
umph or utter disaster in a project.
TIP: HTTP Server Rate Limiting

It’s possible to use nginx or apache httpd for rate limiting. The upside is faster per-
formance. The downside is that it removes this functionality from the Python code.
For a python-based system, DRF provides some basic tools that can be built upon.
ä nginx.com/blog/rate-limiting-nginx/
ä httpd.apache.org/docs/2.4/mod/mod_ratelimit.html
ä django-rest-framework.org/api-guide/throttling/
#setting-the-throttling-policy
17.6.3 Rate Limiting Can Be a Business Plan

Imagine we launch an API-based startup that lets users add images of toppings to images
of ice cream. We know that everyone will want to use this API, and come up with several
tiers of access that we tie into pricing:
Developer tier is free, but only allows 10 API requests per hour.
One Scoop is $24/month, allows 25 requests per minute.
Two Scoops is $79/month, allows 50 requests per minute.
Corporate is $5000/month, allows for 200 requests per minute.
This method is called Bucket-Based Pricing. Now all we have to do is get people to use
our API.
TIP: Consider Usage-Based Pricing Instead of Rate Limit-Based

Pricing
Randall Degges makes a strong argument that the business plan of bucket based
pricing we suggest in this section is not a good business plan. He advocates unlimited
usage from the beginning, where API users are charged in volume, with the price
per unit going down as usage increases. His version of our chart might then be:
ä First 1000 requests made per month are free
ä 1001 to 100,000 requests made per month cost 0.4 cents each
ä 100,001 to 200,000 requests made per month cost 0.3 cents each
ä 200,001+ requests made per month cost 0.2 cents each
The advantage of Usage-Based Pricing is that it incentives volume over trying to

17.7: Advertising Your REST API
fit into buckets or tiers. The downside is it will require more work to set up this
billing plan in your project. Third-party services like Stripe and Paddle can add fees
or additional amounts to subscriptions, so it’s possible to do in common payment
systems.
Reference: rdegges.com/2020/the-only-type-of-api-services-ill-use
17.7 Advertising Your REST API

Let’s assume we’ve built our REST API and want outside coders and companies to use it.
How do we go about doing that?
17.7.1 Documentation
The most important thing to do is to provide comprehensive documentation. The eas-
ier to read and understand the better. Providing easy-to-use code examples is a must.
You can write it from scratch or use auto-documentation tools provided by django-
rest-framework itself and various third-party packages (django-rest-framework.org/
topics/documenting-your-api/#third-party-packages. You can even embrace
commercial documentation generation services like readthedocs.com and swagger.io.
Some of the material in Chapter 25: Documentation: Be Obsessed might prove useful for
forward-facing REST API documentation.
17.7.2 Provide Client SDKs

Something that may help spread use of your API is to provide a software development
kits (SDK) for various programming languages. The more programming languages covered
the better. For us, we’ve found the must-have languages include Python, JavaScript, Ruby,
PHP, Go, and Java.
In our experience, it’s a good idea to write at least one of these libraries ourselves and create
a demo project. The reason is that it not only advertises our API, it forces us to experience
our API from the same vantage point as our consumers.
Fortunately for us, thanks to the underlying OpenAPI document object model of
(openapis.org/), DRF provides JSON Hyperschema-compatible functionality. When
this format is followed for both the client- and server-side, OpenAPI allows for writing
dynamically driven client libraries that can interact with any API that exposes a supported
schema or hypermedia format.
These tools should work immediately with most DRF-powered APIs:

ä Command line client

https://www.django-rest-framework.org/api-guide/schemas/
#generating-an-openapi-schema
ä Multi-language Client SDK generator
github.com/OpenAPITools/openapi-generator
ä swagger-cli for use with JavaScript
npmjs.com/package/swagger-cli
For building Python-powered client SDKs, reading Section 23.9: Releasing Your Own
Django Packages might prove useful.
17.8 Additional Reading

We highly recommend reading the following:
ä en.wikipedia.org/wiki/REST
ä en.wikipedia.org/wiki/List_of_HTTP_status_codes
ä github.com/OAI/OpenAPI-Specification
ä jacobian.org/writing/rest-worst-practices/
17.9 Other Approaches for Crafting APIs

For the reasons explained at the beginning of this chapter, we recommend Django Rest Frame-
work. However, should you choose not to use DRF, consider the following approaches:
17.9.1 CBV Approach: JsonResponse with View

It is possible to use Django’s built-in django.http.JsonResponse class, which is a sub-
class of django.http.HttpResponse , on django.views.generic.View. This sup-
ports the full range of HTTP methods but has no support for OpenAPI. . This is proven
to work with async views.
Example 17.16: Simple JsonResponse View
class FlavorApiView(LoginRequiredMixin,View):
def post(self, request, *args, **kwargs):
# logic goes here
return JsonResponse({})
def get(self, request, *args, **kwargs):

# logic goes here
def put(self, request, *args, **kwargs):

17.10: Summary
# logic goes here

def delete(self, request, *args, **kwargs):

# logic goes here
17.9.2 FBV approach: django-jsonview

While you can use DRF with function-based views, that approach doesn’t have all the fea-
tures of CBV-based DRF. A simpler approach is to use django-jsonview. The downside is
that when you get into the full range of HTTP methods and more complex API designs,
we’ve found in Django FBVs become a hindrance due to their lack of focus on building
APIs or support for OpenAPI.
17.9.3 django-tastypie
django-tastypie is a mature API framework that implements its own class-based view sys-
tem. Predating Django REST Framework by 3 years, it’s a feature-rich, mature, powerful,
stable tool for creating APIs from Django models.
django-tastypie.readthedocs.io/
Figure 17.2: A tasty pie is one filled with ice cream.
17.10 Summary
In this chapter we covered:
ä Why you should use Django Rest Framework

ä Basic REST API concepts and how they relate to Django Rest Framework
ä Security considerations
ä Grouping strategies
ä Simplification strategies

ä Fundamentals of basic REST API design

ä Alternatives to Django REST Framework
Coming up next, we’ll go over the other side of REST APIs in Chapter 19: JavaScript and
Django.

18 | Building GraphQL APIs With
Django
WARNING: This Chapter is in Progress

With the Async Views still not in core, the landscape for Django and GraphQL
remains incomplete. We also did not have enough time to finish consolidating our
thoughts.
Therefore, in the days to come we will expand on the material here. We are open
to suggestions on topics and items to cover, please submit them to github.com/
feldroy/two-scoops-of-django-3.x/issues
Started in 2012, GraphQL is a newcomer to the world of API formats. Unlike REST,
GraphQL comes with schemas, types, and one of our favorite features is a a built-in method
to handle real-time update (Subscriptions). Clients of an API specify the data they want,
and can easily figure out queries because the schemas and types makes it very easy to build
instrospection tools. GraphQL responses are easily serialized to JSON or YAML. For these
reasons and more, GraphQL has seen rapid growth in the past few years.
If you want to learn GraphQL, we recommend the resources at graphql.org: graphql.org/

learn
Because GraphQL comes with a query language and types named in clear English terms
we’ve found it easier to teach beginners how to use GraphQL than REST.
18.1 Dispelling the Performance Myth

A lot of people, upon first learning about how GraphQL clients define the data they want
to receive, raise concerns about performance. The specific issue raised is how can one cache
the data raised by a query if each and every access request could be different?

Chapter 18: Building GraphQL APIs With Django
GraphQL Method REST Method Action

query GetRecords() GET /records Read-only Query
subscription GetRecords() no equivalent Open websocket and up-
date on any changes
query GetRecord(id: X) GET /record/:id Read-only Query
subscription GetRecord(id: GET /record/:id Open websocket and up-
X) date on any changes
mutation CreateRecord() POST /records/ Create record
mutation UpdateRe- PUT /records/:id Update record
cord(id: X)
mutation DeleteRecord(id: DELETE /records/:id Delete record
X)
There are several things that destroy this myth:
1 By specifying only the data they need rather than accepting large REST-style re-
sponses, clients consume significantly less data. This can have a dramatic impact on
both server and database overhead.
2 Much like with a REST API, common acces patterns can still be identified and han-
dled better through caching, indexing, or code optimization.
Do keep in mind we’re not saying that GraphQL APIs are immune to bottlenecks, we’re
just saying that they are at least as manageable as REST APIs, if not more so due to the
lower bandwidth.
Our preferred library for building GraphQL APIs with Django is Ariadne. Here’s why:
ä Fully asynchronous when used with ASGI and Channels, allowed real-time updates
from Django projects via GraphQL Subscriptions.
ä If you aren’t ready for async yet, you can still run under WSGI.
ä Schemas comes first, meaning you are forced to spec out your design before starting to
code. Since GraphQL schemas carry so much weight, this is a very powerful feature.
ä Creating queries and mutations requires following a simple and lightweight API.
ä Supports Apollo Federation, a specification for composing multiple GraphQL ser-
vices into one GraphQL service.
PACKAGE TIP: What About Graphene?

Graphene was the first mainstram Python library to support the GraphQL spec-
ification, and it even comes with a Django integration. The ability to use Django
forms or serializers in nodes is a definate bonus. However, while we respect the ef-
fort put into it, we have found its shortcomings insurmountable for our own projects.

18.2: GraphQL API Architecture
Specifically:
ä Graphene-Django doesn’t support GraphQL Subscriptions, meaning to get
real-time updates one is forced to either poll the API from the client or use a
separate websocket process. Either is an unnacceptable burden and one that
we hope Django’s new Async Views will empower the library maintainers to
correct.
ä On other platforms, such as Apollo server, GraphQL APIs require little code
to implement. In comparison, to us Graphene feels heavyweight.
Reference: docs.graphene-python.org/projects/django/
18.2 GraphQL API Architecture

Building simple, quick GraphQL APIs is easy with Ariadne, but extending and maintaining
it to match your project’s needs takes a bit more thought. This is usually where people get
hung up on API design. This section illustrates some things to keep in mind.
18.2.1 Don’t Use Sequential Keys as Public Identifiers

Sequential keys, such as what Django provides as a default as model primary keys, can be
a security concern if used publicly. We cover this in-depth at Section 28.28: Never Display
Sequential Primary Keys.
In our example, we’re going to use the model’s UUID rather than the model’s primary key
to look up our records. We always try to avoid using sequential numbers for lookups.
18.2.2 Use Consistent API Module Naming

Just like anything else, how things are named needs to be consistent across a project. Here’s
two patterns we’ve explored.
The first is a simple GraphQL architecture suitable for most small-to-mid sized applications.
Because Ariadne’s API is lightweight, we’ve found it’s workable to write all queries and
mutations in one schemas.py module.
Example 18.1: Simple GraphQL Architecture
config/
├── schema.py # imports forms & models from flavors app
├── settings/
├── urls.py
flavors/

├── __init__.py
├── app.py
├── forms.py
├── models.py
Sometimes a single schemas.py module gets too big for comfort. In which case breaking it
up across apps has worked well. Queries and mutations are moved to individual apps, then
imported into the core schemas.py module where make_executable_schema() can be
called on it.
Example 18.2: Complex GraphQL Architecture
config/
├── schema.py # imports queries/mutations from flavors app
├── settings/
├── urls.py
flavors/
├── __init__.py
├── app.py
├── api/
│ ├── queries.py # imports models
│ ├── mutations.py # imports forms and models
├── forms.py
├── models.py
Please observe the following about both architectures:
ä We’re still leaning on Django forms and models

ä We stick to intuitive, consistent naming patterns.
18.2.3 Try to Keep Business Logic Out of API Views

Regardless if your API is small or large, it’s a good idea to keep your logic where it belongs.
Validation should be in forms (or DRF serializers if you prefer them) and database handling
should be in models.
It’s easy with Ariadne to start dumping business logic into function or class definitions.
Avoid this anti-pattern, it makes it harder to test, upgrade, or reuse logic when it comes
time to release APIv2.

18.3: Shutting Down an External API
18.2.4 Test Your API

We’ve found that the best way to test our Ariadne-powered GraphQL is to use
Django’s built-in RequestFactory system. Here is how Ariadne tests it’s Django integra-
tion: github.com/mirumee/ariadne/blob/master/tests/django/test_query_
execution.py
18.2.5 Version Your API

In progress
18.2.6 Be Careful With Customized Authentication Schemes

In progress
18.3 Shutting Down an External API

In progress


19 | JavaScript and Django
This chapter is about JavaScript in the context of Django. Here are the most common ways
JavaScript is used in tandem with Django:
ä To provide the entire frontend and consume REST/GraphQL APIs served by

Django. This is the Single Page App approach often executed with React, Vue, and
other frameworks.
ä To enhance templates written in either DTL or Jinja2 served by Django.
ä To make the Django admin useful and efficient. This is important to remember when
APIs created with Django are serving native apps, even then JavaScript is often still
part of projects.
This chapter will address the following topics:
ä Section 19.1: Popular JavaScript Approaches

ä Section 19.2: Consuming Django-served APIs with JavaScript
ä Section 19.3: Real-Time Woes a.k.a. Latency
ä Section 19.4: Using JavaScript with Templates Served by Django
ä ??: ??
19.1 Popular JavaScript Approaches

With the advent of faster JavaScript engines and a maturation of the associated community,
there has been a rise in new JavaScript frameworks that are designed for integration with
REST or GraphQL APIs. The three most popular ones in 2020 seem to be:
React.js facebook.github.io/react/
A JavaScript framework and ecosystem created and maintained by Facebook. De-
signed for creation of HTML, iOS, and Android applications. React is typically used
for Single Page Apps, but can be used to enhance existing pages.
Vue.js vuejs.org
Vue is designed to be incrementally adoptable. While its ecosystem isn’t as large as

Chapter 19: JavaScript and Django
React.js, the simpler data model is considered easier to learn. In the Django world,
Vue is frequently used to enhance existing pages or for building of Single Page Apps
frontends.
Angular angular.io
Angular is a typescript-only framework open sourced by Google. Within the Django
community it is not as popular as React or Vue.
Reactisso2019.HTML+minimumJSis2020
– David Heinemeier Hansson, Founder of Rails, Basecamp, and Hey
In addition to these comprehensive Single Page Application frameworks the approach of

using serverside rendered HTML templates remains popular. While this might seem like a
step backwards, new approaches to this older architecure has demonstrated that it remains
a viable approach. Sites like hey.com, basecamp.com, and the forthcoming Writernaut
project demonstrate SPA-like speed and power using traditional HTML templates.
Vanilla JavaScript developer.mozilla.org/en-US/docs/Web/JavaScript

Vanilla JavaScript is plain JavaScript without any additional libraries like React,
Vue, or jQuery. There was a time when such libraries were absolutely necessary,
but JavaScript in the browser has become a powerful, feature-rich tool for building
projects.
Small, Focused Libraries + Vanilla JavaScript While Vanilla JavaScript’s interaction with
the DOM has grown in recent years, there are a number of useful libraries that can
empower developers and reduce code complexity. Examples include:
ä Alpine.js github.com/alpinejs/alpine is a rugged, minimal framework
for composing JavaScript behavior in markup. The syntax is Intentionally almost
identical to Vue, giving the power of Vue without the associated heavyweight
architecture.
ä Turbolinks github.com/turbolinks/turbolinks Uses HTML to render
views on the server side and link to pages as usual. When users follow a link,
Turbolinks automatically fetches the page, swaps in its <body>, and merges its
<head>, all without incurring the cost of a full page load. Getting it running
Django isn’t trivial, but can dramatically speed up traditional server side ren-
dered template sites.
jQuery jquery.com
Once the backbone of the JavaScript world, so much of functionality JQuery provides
has now been factored into Vanilla JavaScript it is rarely used in new projects. How-
ever, legions of existing Django projects use it extensively, and will continue to need
to be maintained or upgraded to other tools for years to come.

19.1: Popular JavaScript Approaches
These five options can really improve what we like to call the ‘immediate user experience’.
However, with every good thing there are always things to consider and things to do. Here
are a number of anti-patterns that we’ve discovered when it comes to frontend projects
consuming APIs for content.
19.1.1 Building Single Page Apps When Multi-Page Apps Suffice

Single-page apps with frameworks like React, Vue, and Angular are fun to build, but does a
traditional CMS-site need to be one? Certainly the content pages can include API-powered
editing controls, but when building this kind of site, there is something to be said for tradi-
tional HTML pages.
For example, our health provider in 2017 had a SPA-style site. It was lovely. The way every-
thing moved together was a marvel. And it was completely useless when we had to do any
kind of comparative research.
The worst example of the site was when the search system returned a list of doctors we
couldn’t easily compare them. When we clicked on one for more information, their data was
in a sliding modal. We couldn’t right-click and open several on independant tabs as doing
so just took us to the root search page. We could print or email yourself the information on
individual doctors, but PDFs and email are awful comparison tools compared to hopping
between tabs.
What the site should have provided is individual domain references (i.e. URLs) for each
doctor. Either parsed by the server on the back end or even by JavaScript URL management
on the front end. This isn’t hard to do, yet it remained a painfully common issue until we
went to another healthcare provider.
19.1.2 Upgrading Legacy Sites

Unless the entire site is being scrapped for a new version, don’t upgrade the whole front-end
at once.
When working with legacy projects, it’s often easier to add new features as single-page
apps. This allows for the maintainers of the project to deliver improved experiences with
new features, while preserving the stability of the existing code base. A good example of
this might be adding a calendar application to an existing project. This is easily possible
with Vue and to a lesser extent with React.

19.1.3 Not Writing Tests

When we first begin working in a new language or framework, including client-side
JavaScript, it’s tempting to skip the tests. In a word, don’t. Working in the client is get-
ting more complicated and sophisticated every year. Between evolving client-side standards,
things are simply not as readable there as on the server side.
We cover Django/Python testing in Chapter 24: Testing Stinks and Is a Waste of Money!.

A good reference for JavaScript testing is stackoverflow.com/questions/300855/
javascript-unit-test-tools-for-tdd
19.1.4 Not Understanding JavaScript Memory Management

Single-page apps are great, but the complex implementations where users keep them open
constantly will hold objects in the browser for a very long time. Eventually, if not managed,
this can cause browser slowdowns and crashes. Each JavaScript framework comes with tools
or advice on how to handle this potential problem, and it’s a good idea to know the recom-
mended approach.
19.1.5 Storing Data in the DOM When It’s Not jQuery

After years of using jQuery, some of us have grown used to using DOM elements to store
data (especially Daniel). However, when using other JavaScript frameworks this isn’t ideal.
They have their own mechanisms for handling client data, and by not following them we
risk losing out on some of the features promised by these frameworks.
We recommend looking up the data management methods for your chosen JavaScript frame-
work and embracing them as deeply as possible.
19.2 Consuming Django-served APIs with JavaScript

Now that we’ve covered both creating REST APIs and template best practices, let’s combine
them. In other words, these are best practices for using Django-powered tools to display
content to the end user in the browser using content managed by REST/GraphQL APIs
and presented by modern JavaScript frameworks.
19.2.1 Learn How to Debug the Client

Debugging client-side JavaScript is a lot more than simply writing console.log() and
console.dir() statements. There are a number of tools for debugging and finding errors,
and some of them are specifically written for particular JavaScript frameworks. Once a tool
is chosen, it’s an excellent idea to take a day to learn how to write client-side tests.
Reference material:

19.2: Consuming Django-served APIs with JavaScript
ä developers.google.com/web/tools/chrome-devtools
ä developer.mozilla.org/en-US/docs/Mozilla/Debugging/Debugging_
JavaScript
Figure 19.1: Server-side vs. client-side ice cream.
19.2.2 When Possible, Use JavaScript-Powered Static Asset Preproces-

sors
Up until about 2017, we used Python everywhere, including JavaScript and CSS minifica-
tion. However, the JavaScript community is maintaining their versions of these tools better
than the Python community. That’s perfectly okay, because since they’ve done the work on
this part of the toolchain, we can focus on other things.
As we write this, the most commonly used tool for this kind of work is webpack. Used by
both React, Vue, and every major framework, webpack bundles browser-based scripts to be
served as assets. While we prefer to use the stock webpack setup provided by Vue or React
CLI tools, understanding webpack is a very powerful skill to have.
References:
ä webpack.js.org
ä github.com/owais/django-webpack-loader - Owais Lone’s Django package
for transparently using webpack with Django

19.3 Real-Time Woes a.k.a. Latency

Let’s say we’ve put together a well-designed, well-indexed, well-cached real-time project
with the widest bandwidth piping content to the world. We can handle any load, and our
test users applaud the speed and functionality of the project. Things look great, and we look
forward to bonuses and raises.
Then the complaints from the other side of the planet start coming in about the slow speed
of the application. Our effort isn’t ‘real-time’ to any of a potentially large block of users and
our client/boss is really unhappy.
Congratulations, we’ve just hit the speed of light!
This isn’t a joke, it’s a very real problem. Here, Django isn’t the problem. Instead, it’s physics.
The time it takes for HTTP requests to transmit back and forth across half the circumference
of the planet is noticeable to human beings. Add in server-side and client-side processing,
and we risk alienating potential or existing users.
Also, keep in mind that even the fastest local connections have hiccups and slow-downs. So
it’s not uncommon for ‘real-time’ applications to have ways to handle this sort of behavior.
19.3.1 Solution: Mask the Latency With Animations

One of the more common fixes is to have JavaScript-powered animation distract the user
from latency issues. We encounter this every time we use a single page app with an attractive
interface, including all modern web-based email clients.
19.3.2 Solution: Fake Successful Transactions

Another solution involves processing the request on the client-side as if the request success-
fully made it to the server. We’ll need to include client-side logic to handle failures, but
JavaScript frameworks handling HTTP requests are asynchronous, making this feasible,
albeit possibly complicated.
If you’ve ever suddenly discovered that your cloud-based spreadsheet hadn’t saved the data
entered for the past 30 seconds, you’ve uncovered this kind of JavaScript powered trickery
in action. As this can be very frustrating, some online tools upon detecting a connection
failure, disallow further data entry.
19.3.3 Solution: Geographically Based Servers

Geographically-based servers across all seven continents is an option. However, for Django
this is not trivial to implement, not at the programming or database level. It requires a

19.3: Real-Time Woes a.k.a. Latency
significant volume of skills and expertise that’s outside the scope of this book.
If you have the time and budget, this can be an exciting avenue to explore and we encour-
age it. However, unless you’ve done this before there is a good chance you are going to
underestimate the effort involved.
19.3.4 Solution: Restrict Users Geographically

Sometimes we just don’t have a choice. Perhaps our application is too reliant on ‘real-time’
performance and geolocating servers might be outside the budget. We might make some
people unhappy, but that can be mitigated to some degree by saying things like, ‘Support in
your country is coming soon!’
19.3.5 AJAX and the CSRF Token

Django’s CSRF protection appears to be an inconvenience when writing AJAX calls from
the browser to the server. If you use AJAX with Django, especially with REST APIs,
you may discover that triggering the CSRF token validation blocks your ability to POST,
PATCH, or DELETE data to your API. However, it’s part of what makes Django secure,
don’t disable it!
Our answer to overcoming this hurdle to read and understand the AJAX portion of the
CSRF documentation of Django, listed in the resources below. It comes with sample
JavaScript code that you can put into your projects.
References:
ä docs.djangoproject.com/en/3.2/ref/csrf/
ä github.com/jazzband/dj-rest-auth This is a proven authentication library for
DRF that depending on circumstances can be used without CSRF.
ä ??: ??
WARNING: Don’t Use AJAX as an Excuse to Turn Off CSRF

Django core developer Aymeric Augustin says, “...CSRF protection is almost always
disabled because the developers couldn’t quite figure out how to make it work... if
it accepts cookie authentication.”
Unless using planning to build a machine-to-machine API, don’t build a site with
disabled CSRF. If it’s not possible to figure out how to make it work, ask for help.
No one’s going to make fun of someone trying to make their site more secure.

19.4 Using JavaScript with Templates Served by Django

19.4.1 JavaScript Can Go In the Header Again
For years it was advantageous to put links to scripts and <script> tags at the bottom of a
page body. Modern browsers are much better at parsing how pages load, so this technique
is no longer necessary.
19.4.2 Use JSON Encoding for Data Consumed by JavaScript

Rely on JSON encoding rather than finding ways to dump Python structures directly to
templates. It’s not just easier to integrate into client-side JavaScript, it’s safer. To do this,
always use the json_script filter.
Example 19.1: Properly JSON encoding Data for JavaScript
{{ page_data|json_script:"page-data" }}
<script>
var data = JSON.parse(document.getElementById('page').textContent);
injectNameIntoDiv('scoopName', data.scoop.name);
</script>
Don’t do this, which must we admit to having done for years.
Example 19.2: Improperly JSON encoding Data for JavaScript
// Rendering a value from the Django template

// Hoping no one injected anything malignant
injectNameIntoDiv('scoopName', {{ scoop.name }});
Reference: docs.djangoproject.com/en/3.2/ref/templates/builtins/
#json-script
19.5 Strengthening JavaScript Skills

One of the best things we can do when implementing the consumption of REST APIs
on the client side is to ensure our JavaScript skills are up to par. While Python developers
sometimes like to grumble about JavaScript, it is a very capable language in its own right.
Any responsible web developer will take the time to ramp up their skills so they can reap
the benefits of modern JavaScript.
TODO

19.6: Follow JavaScript Coding Standards
19.5.1 Learn More JavaScript!

There are plenty of resources available for improving your basic JavaScript skills. We list our
favorites at the end of Appendix C: Additional Resources.
19.6 Follow JavaScript Coding Standards

In the case of JavaScript, we advocate the following guides for both front- and back-end
work:
ä Felix’s Node.js Style Guide

nodeguide.com/style.html
ä idiomatic.js
github.com/rwaldron/idiomatic.js
19.7 Summary
TODO


20 | Tradeoffs of Replacing
Core Components
Some people advocate swapping out core parts of Django’s stack for other pieces. Should it
be done?
Short Answer: Don’t do it. Even one of the founders of Instagram (Kevin Systrom) said
to Forbes.com that it’s completely unnecessary (bit.ly/2pZxOBO).
Long Answer: It’s certainly possible, since Django modules are simply just Python mod-
ules. Is it worth it? Well, it’s worth it only if:
ä You are okay with sacrificing some or all of your ability to use third-party Django
packages.
ä You have no problem giving up the powerful Django admin.
ä You have already made a determined effort to build your project with core
Django components, but you are running into walls that are major blockers.
ä You have already analyzed your own code to find and fix the root causes of your
problems. For example, you’ve done all the work you can to reduce the numbers
of queries made in your templates.
ä You’ve explored all other options including caching, denormalization, etc.
ä Your project is a real, live production site with tons of users. In other words,
you’re certain that you’re not just optimizing prematurely.
ä You’ve looked at and rejected adopting a Service Oriented Approach (SOA) for
those cases Django has problems dealing with.
ä You’re willing to accept the fact that upgrading Django will be extremely painful
or impossible going forward.
That doesn’t sound so great anymore, does it?

Chapter 20: Tradeoffs of Replacing Core Components
20.1 The Temptation to Build FrankenDjango

Every few years, a new fad leads waves of developers to replace some particular core Django
component. Here’s a summary of some of the fads we’ve seen come and go.
Figure 20.1: Replacing more core components of cake with ice cream seems like a good idea.
Which cake would win? The one on the right!
20.2 Non-Relational Databases vs. Relational

Databases
Even Django projects that use relational databases for persistent data storage rely on non-
relational databases. If a project relies on tools like Memcached for caching and Redis for
queuing, then it’s using non-relational databases.
The problem occurs when NoSQL solutions are used to completely replace Django’s rela-
tional database functionality without considering in-depth the long-term implications.
20.2.1 Not All Non-Relational Databases Are ACID Compliant

ACID is an acronym for:
Atomicity means that all parts of a transaction work or it all fails. Without this, you risk
data corruption.
Consistency means that any transaction will keep data in a valid state. Strings remain
strings and integers remain integers. Without this, you risk data corruption.
Isolation means that concurrent execution of data within a transaction will not collide or
leak into another transaction. Without this, you risk data corruption.
Durability means that once a transaction is committed, it will remain so even if the
database server is shut down. Without this, you risk data corruption.

20.2: Non-Relational Databases vs. Relational
Databases
Fad Reasons
For performance reasons, replacing the Not okay: “I have an idea for a social net-
database/ORM with a NoSQL database work for ice cream haters. I just started
and corresponding ORM replacement. building it last month. It must scale to bil-
lions!
Okay: “Our site has 50M users and I’m

hitting the limits of what I can do with
indexes, query optimization, caching, etc.
We’re also pushing the limits of our Post-
gres cluster. I’ve done a lot of research on
this and am going to try storing a simple
denormalized view of data in Cassandra to
see if it helps. I’m aware of the CAP the-
orem (en.wikipedia.org/wiki/CAP_
theorem), and for this view, eventual con-
sistency is fine.”
For data processing reasons, replacing the Not okay: “SQL Sucks! We’re going with
database/ORM with a NoSQL database a document-oriented database like Mon-
and corresponding ORM replacement. goDB!”
Okay: “PostgreSQL’s and MySQL’s

JSON datatypes replicate nearly every
aspect of MongoDB’s data storage system.
Yes, MongoDB’s has built-in MapReduce
functionality, but it’s easier to execute that
in a task queue.”
Replacing Django’s template engine with Not okay: “I read that Jinja2 is faster. I
Jinja2, Mako, or something else. don’t know anything about caching or op-
timization, but I need Jinja2!”
Not okay: “I hate having logic in Python

modules. I want logic in my templates!”
Okay: “I have a small number of views

which generate 1MB+ HTML pages de-
signed for Google to index. I’ll use
Django’s native support for multiple tem-
plate languages to render the 1MB+ sized
pages with Jinja2, and serve the rest with
Django Template Language.”
Table 20.1: Fad-based Reasons to Replace Components of Django

Did you notice how each of those descriptions ended with ‘Without this, you risk data corrup-
tion.’? This is because for many NoSQL engines, there is little-to-no mechanism for ACID
compliance. It’s much easier to corrupt the data, which is mostly a non-issue for things like
caching but another thing altogether for projects handling processing of persistent medical
or e-commerce data.
20.2.2 Don’t Use Non-Relational Databases for Relational Tasks

Imagine if we were to use a non-relational database to track the sale of properties, prop-
erty owners, and how property laws worked for them in 50 US states. There are a lot of
unpredictable details, so wouldn’t a schemaless datastore be perfect for this task?
Perhaps...
We would need to track the relationship between properties, property owners, and laws of
50 states. Our Python code would have to maintain the referential integrity between all the
components. We would also need to ensure that the right data goes into the right place.
For a task like this, stick with a relational database.
20.2.3 Ignore the Hype and Do Your Own Research

It’s often said that non-relational databases are faster and scale better than relational
databases. Whether or not this is true, don’t blindly swallow the marketing hype of the
companies behind any particular alternative database solution.
Instead, do as we do: search for benchmarks, read case studies describing when things went
right or wrong, and form opinions as independently as possible.
Also, experiment with unfamiliar NoSQL databases on small hobby side projects before
you make major changes to your main project infrastructure. Your main codebase is not a
playground.
Lessons learned by companies and individuals:
ä Pinterest: medium.com/@Pinterest_Engineering/
stop-using-shiny-3e1613c2ce14
ä Dan McKinley while at Etsy: mcfunley.com/
why-mongodb-never-worked-out-at-etsy
ä When to use MongoDB with Django daniel.feldroy.com/
when-to-use-mongodb-with-django.html
250 Please submit

Prepared exclusively issues to github.com/feldroy/two-
for LAKLAK scoops- of- django- 3.x/issues
LAKLAK ([email protected]) Transaction: 11322
20.3: What About Replacing the Django Template Language?
20.2.4 How We Use Non-Relational Databases With Django

This is how we prefer to do things:
ä If we use a non-relational data store, limit usage to short-term things like caches,
queues, and sometimes denormalized data. But avoid it if possible, to reduce the
number of moving parts.
ä Use relational data stores for long-term, relational data and sometimes denormalized
data (PostgreSQL’s array and JSON fields work great for this task).
For us, this is the sweet spot that makes our Django projects shine.
20.3 What About Replacing the Django Template

Language?
We advocate the practice of sticking entirely to the Django Template Language (DTL) with
the exception of rendered content of huge size. However, as this use case is now covered by
Django’s native support of alternate template systems, we’ve moved discussion of this topic
to Chapter 16: Django Templates and Jinja2.
20.4 Summary
Always use the right tool for the right job. We prefer to go with stock Django components,
just like we prefer using a scoop when serving ice cream. However, there are times when
other tools make sense.
Just don’t follow the fad of mixing vegetables into your ice cream. You simply can’t replace
the classic strawberry, chocolate, and vanilla with supposedly “high-performance” flavors
such as broccoli, corn, and spinach. That’s taking it too far.
Please submit
LAKLAK ([email protected]) Transaction: 11322 251
252 Please submit

21 | Working With the Django Admin
When people ask, “What are the benefits of Django over other web frameworks?” the admin is
what usually comes to mind.
Imagine if every gallon of ice cream came with an admin interface. You’d be able to not
just see the list of ingredients, but also add/edit/delete ingredients. If someone was messing
around with your ice cream in a way that you didn’t like, you could limit or revoke their
access.
Figure 21.1: Chocolate chip ice cream with an admin interface.
Pretty surreal, isn’t it? Well, that’s what web developers coming from another background
feel like when they first use the Django admin interface. It gives you so much power over
your web application automatically, with little work required.
21.1 It’s Not for End Users

The Django admin interface is designed for site administrators, not end users. It’s a place
for your site administrators to add/edit/delete data and perform site management tasks.
Please submit
Chapter 21: Working With the Django Admin
Although it’s possible to stretch it into something that your end users could use, you really
shouldn’t. It’s just not designed for use by every site visitor.
21.2 Admin Customization vs. New Views

It’s usually not worth it to heavily customize the Django admin. Sometimes, creating a
simple view or form from scratch results in the same desired functionality with a lot less
work. We’ve always had better results with creating custom management dashboards for
client projects than we have with modifying the admin to fit the need of the client.
21.3 Viewing String Representations of Objects

The default admin page for a Django app shows a list of generic looking objects like this:
Figure 21.2: Admin list page for an ice cream bar app.
That’s because the default string representation of an IceCreamBar object is “IceCreamBar

object”. Wouldn’t it be helpful to display something better?
Figure 21.3: What? An admin interface for ice cream bars?
254 Please submit

21.3: Viewing String Representations of Objects
21.3.1 Using __str__()

Implementing __str__() is straightforward:
Example 21.1: String Representation of Objects
from django.db import modelss
class IceCreamBar(models.Model):
name = models.CharField(max_length=100)
shell = models.CharField(max_length=100)
filling = models.CharField(max_length=100)
has_stick = models.BooleanField(default=True)
def __str__(self):
return self.name
The result is as follows:
Figure 21.4: Improved admin list page with better string representation of our objects.
It’s more than that. When you’re in the shell, you see the better string representation:
Example 21.2: List of Ice Cream Bar Types
>>> IceCreamBar.objects.all()
[<IceCreamBar: Vanilla Crisp>, <IceCreamBar: Mint Cookie Crunch>,
<IceCreamBar: Strawberry Pie>]
Please submit
The __str__() method is called whenever you call str() on an object. This occurs in the
Django shell, templates, and by extension the Django admin. Therefore, try to make the
results of __str__() nice, readable representation of Django model instances.
21.3.2 Using list_display

If you want to change the admin list display in a way that isn’t quite a string representation
of the object, then use list_display.
Example 21.3: Admin List Display
from django.contrib import admin
from .models import IceCreamBar
@admin.register(IceCreamBar)
class IceCreamBarModelAdmin(admin.ModelAdmin):
list_display = ('name', 'shell', 'filling')
The result with the specified fields:
Figure 21.5: Further improvements to the admin list page.
21.4 Adding Callables to ModelAdmin Classes

You can use callables such as methods and functions to add functionality to the Django
django.contrib.admin.ModelAdmin class. This allows you to really modify the list
and display screens to suit your ice cream project needs.
256 Please submit

21.4: Adding Callables to ModelAdmin Classes
For example, it’s not uncommon to want to see the exact URL of a model instance in
the Django admin. If you define a get_absolute_url() method for your model, what
Django provides in the admin is a link to a redirect view whose URL is very different from
the actual object URL. Also, there are cases where the get_absolute_url() method is
meaningless (REST APIs come to mind).
In the example below, we demonstrate how to use a simple callable to provide a link to our
target URL:
Example 21.4: Adding Callables to ModelAdmin Classes
# icecreambars/admin.py
from django.contrib import admin
from django.urls import reverse, NoReverseMatch
from django.utils.html import format_html
from .models import IceCreamBar
@admin.register(IceCreamBar)
class IceCreamBarModelAdmin(admin.ModelAdmin):
list_display = ('name', 'shell', 'filling')
readonly_fields = ('show_url',)
def show_url(self, instance):

url = reverse('icecreambars:ice_cream_bar_detail',
,→ kwargs={'pk': instance.pk})
response = format_html("""<a href="{0}">{0}</a>""", url)
return response
show_url.short_description = 'Ice Cream Bar URL'
Since a picture is worth a thousand words, here is what our callable does for us:
Please submit
Figure 21.6: Displaying URL in the Django Admin.
WARNING: Always Use format_html When Presenting Data

From Users
Any time data is presented in the admin that includes HTML tags, use
format_html to ensure that arguments are escaped during the rendering process.
This is explained more at Section 28.9.1: Use format_html Over mark_safe.
21.5 Be Aware of the Complications of Multiuser Environ-

ments
Nothing in the Django admin locks records to a particular staff- or admin-level user. While
this is fine for a project with a single person with admin-level access, on a multi-user project
it can be a very serious problem. Here is what happens:
1 Via the Django admin, Daniel edits the record for “Peppermint Sundae” ice cream
bar. He starts to make changes. He gets a phone call from the marketing officer of
Icecreamlandia and leaves his screen open.
2 In the meantime, Audrey decides to modify “Peppermint Sundae” ice cream bar. She
spends ten minutes making her changes, then saves the her data.
3 Daniel gets off the phone and finally saves his changes. He overwrites Audrey’s
changes.
If you have multiple users with access to the Django admin, you need to be aware of this
258 Please submit

21.6: Django’s Admin Documentation Generator
possibility.
21.6 Django’s Admin Documentation Generator

One of the more interesting developer tools that Django provides is the
django.contrib.admindocs package. Created in an era before the advent of the
documentation tools that we cover in Chapter 25: Documentation: Be Obsessed, it
remains a useful tool.
It’s useful because it introspects the Django framework to display docstrings for project
components like models, views, custom template tags, and custom filters. Even if a project’s
components don’t contain any docstrings, simply seeing a list of harder-to-introspect items
like oddly named custom template tags and custom filters can be really useful in exploring
the architecture of a complicated, existing application.
Using django.contrib.admindocs is easy, but we like to reorder the steps described in

the formal documentation:
1 pip install docutils into your project’s virtualenv.

2 Add django.contrib.admindocs to your INSTALLED_APPS.
3 Add path('admin/doc/', include('django.contrib.admindocs.urls'))
to your root URLConf. Make sure it’s included before the admin/ entry, so that
requests to /admin/doc/ don’t get handled by the latter entry.
4 Optional: Using the admindocs bookmarklets requires the XViewMiddleware to be
installed.
Once you have this in place, go to /admin/doc/ and explore. You may notice a lot of
your project’s code lacks any sort of documentation. This is addressed in the formal doc-
umentation on django.contrib.admindocs: docs.djangoproject.com/en/3.2/
ref/contrib/admin/admindocs/ and our own chapter on Chapter 25: Documenta-
tion: Be Obsessed.
21.7 Using Custom Skins With the Django Admin

Over the years there have been a number of efforts to reskin or theme the Django Admin.
These range from the venerable, stable, and very popular django-grappelli to more recent
up-and-comers. They allow easy-to-hard customization.
PACKAGE TIP: Custom django.contrib.admin Skins

Here are some of the more popular custom skins:
ä django-grappelli is the grand-daddy of all custom Django skins. Stable, ro-
bust, and with a unique but friendly style.
Please submit
ä django-suit is built using the familiar Bootstrap front-end framework.

ä jet-admin is an API-based Admin Panel Framework.
A more complete list can be found at
djangopackages.org/grids/g/admin-styling/.
Django has a gigantic community, so why aren’t there more skins?
It turns out that besides the most basic CSS-based modifications, creating custom Django
themes is very challenging. For anyone who has delved into the source code for these
projects, it’s clear that custom admin skins require arcane code to account for some of the
idiosyncrasies of django.contrib.admin.
Patrick Kranzlmueller, maintainer of django-grappelli, goes into great detail in his article
on the subject, ‘A Frontend Framework for the Django Admin Interface’, which you can
read at the link below:
ä sehmaschine.net/blog/django-admin-frontend-framework.
Here are some tips when working with custom django.contrib.admin skins:
21.7.1 Evaluation Point: Documentation is Everything

As mentioned earlier, writing a custom skin for django.contrib.admin is hard. While
the successful skins are relatively easy to add to a project, it’s the edge cases (invariably
involved in extending the ModelAdmin object) that can hurt.
Therefore, when evaluating one of these projects for use on a project, check to see how far
the documentation goes beyond installation instructions.
21.7.2 Write Tests for Any Admin Extensions You Create

For our purposes, we’ve found that while clients enjoy the more modern themes, you
have to be careful of how far you extend these admin skins. What works great in vanilla
django.contrib.admin can break in a custom skin. Since the custom skins have to wrap
portions of django.contrib.admin abstractions in curious ways, debugging these prob-
lems can prove to be a mind-numbing nightmare.
Therefore, if you use a custom skin, the best practice is to write tests of the admin, especially
for any customization. Yes, it is a bit of work up front, but it means catching these bugs much,
much earlier.
260 Please submit

21.8: Secure the Django Admin
For more on testing, see our writings on testing in Chapter 24: Testing Stinks and Is a
Waste of Money!.
21.8 Secure the Django Admin

Since the Django admin gives your site admins special powers that ordinary users don’t have,
it’s good practice to make it extra secure.
21.8.1 Change the Default Admin URL

By default, the admin URL is yoursite.com/admin/. Change it to something that’s long and
difficult to guess.
TIP: Jacob Kaplan-Moss Talks About Changing the Admin URL

Django project co-leader Jacob Kaplan-Moss says (paraphrased) that it’s an easy ad-
ditional layer of security to come up with a different name (or even different domain)
for the admin.
It also prevents attackers from easily profiling your site. For example, attackers can
tell which version of Django you’re using, sometimes down to the point-release level,
by examining the HTML content of the login screen for admin/ .
21.8.2 Use django-admin-honeypot

TODO - Ask Derek to update this package
If you’re particularly concerned about people trying to break into your Django site, django-
admin-honeypot is a package that puts a fake Django admin login screen at admin/ and
logs information about anyone who attempts to log in.
See github.com/dmpayton/django-admin-honeypot for more information.
21.8.3 Only Allow Admin Access via HTTPS

This is already implied in Section 28.6: HTTPS Everywhere, but we want to especially em-
phasize here that your admin needs to be TLS-secured. If your site allows straight HTTP
access, you will need to run the admin on a properly-secured domain, adding to the complex-
ity of your deployment. Not only will you need a second deployment procedure, but you’ll
need to include logic in your URLConf in order to remove the admin from HTTP access.
In the experience of the authors, it’s much easier to put the whole site on TLS/HTTPS.
Please submit
Without TLS, if you log into your Django admin on an open WiFi network, it’s trivial for
someone to sniff your admin username/password.
21.8.4 Limit Admin Access Based on IP

Configure your web server to only allow access to the Django admin to certain IP addresses.
Look up the instructions for your particular web server.
ä Nginx instructions tech.marksblogg.com/django-admin-logins.html
An acceptable alternative is to put this logic into middleware. It’s better to do it at the
web server level because every middleware component adds an extra layer of logic wrapping
your views, but in some cases this can be your only option. For example, your platform-as-
a-service might not give you fine-grain control over web server configuration.
21.9 Securing the Admin Docs

Since the Django admin docs give your site admins a view into how the project is con-
structed, it’s good practice to keep them extra-secure just like the Django admin. Borrowing
from the previous section on the Django admin, we advocate the following:
ä Changing the admin docs URL to something besides yoursite.com/admin/doc/.

ä Only allowing admin docs access via HTTPS.
ä Limiting admin docs access based on IP.
21.10 Summary
In this chapter we covered the following:
ä Who should be using the Django admin.

ä When to use the Django admin and when to roll a new dashboard.
ä String representation of objects.
ä Adding callables to Django admin classes.
ä Using Django’s admin docs.
ä Encouraging you to secure the Django admin.
ä Advised on working with custom Django skins.
262 Please submit

22 | Dealing With the User Model
Django comes with a built-in support for user records. It’s a useful feature, doubly so once
you learn how to extend and expand on the basic functionality. So let’s go over the best
practices:
22.1 Use Django’s Tools for Finding the User Model

The advised way to get to the user class is as follows:
Example 22.1: Using get_user_model to the User Record
# Stock user model definition

>>> from django.contrib.auth import get_user_model
>>> get_user_model()
<class django.contrib.auth.models.User>
# When the project has a custom user model definition

>>> from django.contrib.auth import get_user_model
>>> get_user_model()
<class profiles.models.UserProfile>
It is possible to get two different User model definitions depending on the project configu-
ration. This doesn’t mean that a project can have two different User models; it means that
every project can customize its own User model.
22.1.1 Use settings.AUTH_USER_MODEL for Foreign Keys to User

In Django, the official preferred way to attach ForeignKey, OneToOneField, or
ManyToManyField to User is as follows:
Please submit
Chapter 22: Dealing With the User Model
Example 22.2: Using settings.AUTH_USER_MODEL to Define Model Rela-

tions
from django.conf import settings

from django.db import models
class IceCreamStore(models.Model):
owner = models.OneToOneField(settings.AUTH_USER_MODEL)
title = models.CharField(max_length=255)
Yes, it looks a bit strange, but that’s what the official Django docs advise.
Figure 22.1: This looks strange too.
WARNING: Don’t Change settings.AUTH_USER_MODEL!

Once set in a project, changing settings.AUTH_USER_MODEL requires changing
your database schema accordingly. It’s one thing to add or modify User model fields,
it’s another thing to create a whole new User object.
22.1.2 Don’t Use get_user_model() for Foreign Keys to User

This is bad, as it tends to create import loops.
Example 22.3: Using get_user_model() Improperly
# DON'T DO THIS!
from django.contrib.auth import get_user_model
class IceCreamStore(models.Model):
264 Please submit

22.2: Custom User Fields for Django Projects
# This following line tends to create import loops.

owner = models.OneToOneField(get_user_model())
title = models.CharField(max_length=255)
22.2 Custom User Fields for Django Projects

In Django, as long as we incorporate the required methods and attributes, we can create our
own user model with its own fields.
PACKAGE TIP: Libraries for Defining Custom User Models

django-authtools is a library that makes defining custom user models easier. Of
particular use are the AbstractEmailUser and AbstractNamedUser models.
Even if you don’t end up using django-authtools, the source code is well worth
examining.
At this time django-authtools doesn’t support Django 3, even though there is a pull
request adding it. Nevertheless, as mentioned before the code is worth a look.
22.2.1 Option 1: Subclass AbstractUser

Choose this option if you like Django’s User model fields the way they are, but need extra
fields. For what it’s worth, this is the first approach that we look at anytime we start a new
project. When using django-authtools’ base models, forms, and admin objects, we find that
it’s the quickest and easiest way to implement custom user models.
Here’s an example of how to subclass AbstractUser:
Example 22.4: Subclassing of AbstractUser
# profiles/models.py
from django.contrib.auth.models import AbstractUser
class KarmaUser(AbstractUser):
karma = models.PositiveIntegerField(verbose_name='karma',
default=0,
blank=True)
The other thing you have to do is set this in your settings:
Please submit
Example 22.5: Setting AUTH_USER_MODEL
AUTH_USER_MODEL = 'profiles.KarmaUser'
22.2.2 Option 2: Subclass AbstractBaseUser

AbstractBaseUser is the bare-bones option with only 3 fields: password, last_login,
and is_active.
Choose this option if:
ä You’re unhappy with the fields that the User model provides by default, such as
first_name and last_name.
ä You prefer to subclass from an extremely bare-bones clean slate but want to take
advantage of the AbstractBaseUser sane default approach to storing passwords.
If you want to go down this path, we recommend the following reading:
Official Django Documentation Example

docs.djangoproject.com/en/3.2/topics/auth/customizing/
#a-full-example
Source code of django-authtools (Especially admin.py, forms.py, and models.py)
github.com/fusionbox/django-authtools
22.2.3 Option 3: Linking Back From a Related Model

This code is very similar to the pre-Django 1.5 project technique of creating ‘Profile’ models.
Before discarding this approach as legacy, consider the following use cases:
Use Case: Creating a Third Party Package
ä We are creating a third-party package for publication on PyPI.

ä The package needs to store additional information per user, perhaps a Stripe ID or
another payment gateway identifier.
ä We want to be as unobtrusive to the existing project code as possible. Loose coupling!
Use Case: Internal Project Needs
ä We are working on our own Django project.

ä We want different types of users to have different fields.
ä We might have some users with a combination of different user types.
266 Please submit

22.2: Custom User Fields for Django Projects
ä We want to handle this at the model level, instead of at other levels.

ä We want this to be used in conjunction with a custom user model from options #1 or
#2.
Either of these use cases provide motive for the continued use of this technique.
To make this technique work, we continue to use django.contrib.models.User (called

preferably via django.contrib.auth.get_user_model()) and keep your related fields
in separate models (e.g. Profiles). Here’s an example:
Example 22.6: Custom User Profile Examples
# profiles/models.py
from django.conf import settings

from flavors.models import Flavor
class EaterProfile(models.Model):
# Default user profile

# If you do this you need to either have a post_save signal or
# redirect to a profile_edit view on initial login.
user = models.OneToOneField(settings.AUTH_USER_MODEL)
favorite_ice_cream = models.ForeignKey(Flavor, null=True,
,→ blank=True)
class ScooperProfile(models.Model):
scoops_scooped = models.IntegerField(default=0)
class InventorProfile(models.Model):
flavors_invented = models.ManyToManyField(Flavor, null=True,
,→ blank=True)
Using this approach, we can query for any user’s favorite ice cream trivially with the ORM:
user.eaterprofile.favorite_ice_cream. In addition, Scooper and Inventor
Please submit
profiles provide individual data that only applies to those users. Since that data is isolated
into dedicated models, it’s much harder for accidents between user types to occur.
The only downside to this approach is that it’s possible to take it too far in complexity of
profiles or in the supporting code. As always, keep your code as simple and clean as possible.
WARNING: Third-Party Libraries Should Not Be Defining the

User Model
Unless the express purpose of the library is to define custom user models for a project
(à la django-authtools), third-party libraries shouldn’t be using options #1 or #2 to
add fields to user models. Instead, they should rely on option #3.
22.3 Handling Multiple User Types

A common question from Django beginners is how to handle multiple user types. For ex-
ample with ice cream stores, dealing with customer, employee, and owner user types. Often
they think about defining two different user models, which is a dangerous approach to take
- Django isn’t designed to work that way.
Instead, just use one model and mark it appropriately. There’s two primary ways to do this:
22.3.1 Add a User Type Field

This approach assumes different roles have the same data and methods available to them.
In the user model, add a choices field distingishing between types of users. This allows for
role checks across a Django project.
Example 22.7: A Choices-Based User Type Field
class User(AbstractUser):
class Types(models.TextChoices):
EATER = "EATER", "Eater"
SCOOPER = "SCOOPER", "Scooper"
INVENTOR = "INVENTOR", "Inventor"
# What type of user are we?

type = models.CharField(
_("Type"), max_length=50, choices=Types.choices,
,→ default=Types.EATER
)
268 Please submit

22.3: Handling Multiple User Types
Please note that we don’t rely on the multiple use of BooleanField to control roles. That
approach works only in limited circumstances, and becomes confusing as a site grows in size
and complexity over time. Better to use a single field to control the role of a User.
Furthermore, the approach we’re using relies on a CharField. In practice, we tend to use
ForeignKey relations to a Role model. If users need multiple roles, then we will either make
that a ManyToManyField or lean on Django’s built-in Group system. The disadvantage of
using Django’s built-in Group system is the minimal documentation for it.
22.3.2 Add a User Type Field Plus Proxy Models

It is typical for different types of users to have different methods and properties. For example,
a SCOOPER would have a scoop_icecream() method and an EATER would have a
consume() method.
Let’s also make querying for types of user as obvious and explicit as possible. It is easy to
forget to include a filter on a type of user, granting them access to things they should not
have.
Proxy models makes implementing this easy. First, in our model, add a base_type property
and extend the built-in save() method:
Example 22.8: ”The base_type Property”
class User(AbstractUser):
class Types(models.TextChoices):
EATER = "EATER", "Eater"
SCOOPER = "SCOOPER", "Scooper"
INVENTOR = "INVENTOR", "Inventor"
# Ensures that creating new users through proxy models works

base_type = Types.EATER
# What type of user are we?

type = models.CharField(
_("Type"), max_length=50,
choices=Types.choices,
default=Types.EATER
)
# ...
Please submit
def save(self, *args, **kwargs):

# If a new user, set the user's type based off the
# base_type property
if not self.pk:
self.type = self.base_type
return super().save(*args, **kwargs)
In our users/models.py module, underneath the User mode:
Example 22.9: ”Adding the Inventor Proxy Model”
class InventorManager(BaseUserManager):
def get_queryset(self, *args, **kwargs):
results = super().get_queryset(*args, **kwargs)
return results.filter(type=User.Types.INVENTOR)
class Inventor(User):
# This sets the user type to INVENTOR during record creation
base_type = User.Types.INVENTOR
# Ensures queries on the Inventor model return only Inventors

objects = InventorManager()
# Setting proxy to "True" means a table WILL NOT be created

# for this record
class Meta:
proxy = True
# Only inventors get to invent new flavors!

def invent(self):
# Magical custom logic goes Here
return "Delicious!"
As mentioned in the comments, proxy models don’t add fields. What they do create is refer-
ences to a model object on which we can hang custom managers, methods, and properties.
The example queries below will show the power of this approach:
270 Please submit

22.3: Handling Multiple User Types
Example 22.10: Demonstrating the Power of Proxy Models
>>> from users.models import User, Inventor

>>> User.objects.count() # Over 300 million users!
323482357
>>> Inventor.objects.count() # But only 3 inventors
3
>>> # Calling someone as both a User and an Inventor
>>> user = User.objects.get(username='umafeldroy')
>>> user
<User: uma>
>>> inventor = Inventor.objects.get(username='umafeldroy')
>>> inventor
<Inventor: uma>
>>> # Calling a method that's only for inventors
>>> user.invent()
AttributeError
>>> inventor.invent()
'Delicious'
The proxy approach gives us multiple types of users without creating a new User table or
being forced to dramatically extend django.contrib.auth in such a way that we can’t
use third-party libraries.
That proxy models can have their own model managers means that we can have more ex-
plicit queries. This helps prevent mistakes in granting permission. Compare these two code
examples for clarity:
Example 22.11: Explicitly Filtering on Inventor Type
>>> User.objects.filter(type=User.Types.INVENTOR)
>>> Inventor.objects.filter() # Our preference
22.3.3 Adding Extra Data Fields

We’ve found there two ways to handle extra data fields for different user types:
1 Use OneToOneField relations to profile models as described in Section 22.2.3: Op-

tion 3: Linking Back From a Related Model.
2 Put all the fields in the base User model. This approach is simple, but with enough
users and user-specific data can result in the User table slowing. Even without perfor-
Please submit
mance considerations in, there is significant risk of innappropiate data being held by
the wrong user type.
Our preference is the first option, to link back from a related model and combine with proxy
models. Here’s how we do it using all the ”Profile” models from Section 22.2.3: Option 3:
Linking Back From a Related Model.
class Inventor(User):
# ...
objects = InventorManager()
class Meta:
proxy = True
@property
def extra(self):
return self.inventorprofile
class Scooper(User):
# ...
objects = ScooperManager()
class Meta:
proxy = True
@property
def extra(self):
return self.scooperprofile
class Eater(User):
# ...
objects = EaterManager()
class Meta:
proxy = True
@property
def extra(self):
return self.eaterprofile
Now, no matter the user type, the Profile from the One-to-One relationship can be accessed
272 Please submit

22.4: Summary
thus:
ä eater.extra.favorite_ice_cream
ä scooper.extra.scoops_scooped
ä invented.extra.flavors_invented
We really like this approach, it is easy to remember that the relation to the profile table is
accessible via the extra property. It even works with architectures that allow users to have
more than one role.
Proxy models are something typically used sparingly as complex implementations can be
confusing. However, under the right circumstances such as multiple user types, they are
incredibly useful.
22.3.4 Additional Resources on Multiple User Types

ä Daniel Feldroy’s Youtube tutorial on this topic: feld.to/multiple-user-types
ä Official documentation on Proxy Models: docs.djangoproject.com/en/3.2/
topics/db/models/#proxy-models
ä Vitor Freitas’ of ‘Simple is better than Complex article’ on the topic of multiple user
types. Very useful article, even if he doesn’t use proxy models: feld.to/2VSi3My
22.4 Summary
In this chapter we covered the new method to find the User model and define our own
custom ones. Depending on the needs of a project, they can either continue with the current
way of doing things or customize the actual user model. We also covered the use case of
multiple user model types through proxy models.
The next chapter is a dive into the world of third-party packages.
Please submit
274 Please submit

23 | Django’s Secret Sauce:
Third-Party Packages
TODO - Add mention of https://www.python.org/dev/peps/pep-0517/
The real power of Django is more than just the framework and documentation available at
djangoproject.com. It’s the vast selection of third-party Django and Python packages
provided by the open source community. There are many, many third-party packages avail-
able for your Django projects which can do an incredible amount of work for you. These
packages have been written by people from all walks of life, and they power much of the
world today.
Figure 23.1: A jar of Django’s mysterious secret sauce. Most don’t have a clue what this is.
Much of professional Django and Python development is about the incorporation of third-
party packages into Django projects. If you try to write every single tool that you need from
scratch, you’ll have a hard time getting things done.
This is especially true for us in the consulting world, where client projects consist of many
of the same or similar building blocks.
Please submit
Chapter 23: Django’s Secret Sauce: Third-Party Packages
Figure 23.2: The secret is out. It’s just hot fudge.
23.1 Examples of Third-Party Packages

Chapter 37: Appendix A: Packages Mentioned In This Book covers all of the packages
mentioned throughout Two Scoops of Django. This list is a great starting point if you’re
looking for highly-useful packages to consider adding to your projects.
Note that not all of those packages are Django-specific, that means that you can use some of
them in other Python projects. (Generally, Django-specific packages have names prefixed
with “django-” or “dj-”, but there are many exceptions.)
23.2 Know About the Python Package Index

The Python Package Index (PyPI), located at pypi.org/, is a repository of software for
the Python programming language. As of the time this sentence was written, it lists over
100,000 packages, including Django itself.
For the vast majority of Python community, no open source project release is considered
official until it occurs on the Python Package Index.
The Python Package Index is much more than just a directory. Think of it as the world’s
largest center for Python package information and files. Whenever you use pip to install a
particular release of Django, pip downloads the files from the Python Package Index. Most
Python and Django packages are downloadable from the Python Package Index in addition
to pip.
23.3 Know About DjangoPackages.org

Django Packages (djangopackages.org) is a directory of reusable apps, sites, tools and
more for your Django projects. Unlike PyPI, it doesn’t store the packages themselves, in-
stead providing a mix of hard metrics gathered from the Python Package Index, GitHub,
ReadTheDocs, and “soft” data entered by users.
Django Packages is best known as a comparison site for evaluating package features. On
276 Please submit

23.4: Know Your Resources
Django Packages, packages are organized into handy grids so they can be compared against
each other.
Django Packages also happens to have been created by the authors of this book, with con-
tributions from many, many people in the Python community. Thanks to many volunteers
it is continually maintained and improved as a helpful resource for Django users.
23.4 Know Your Resources

Django developers unaware of the critical resources of Django Packages and the Python
Package Index are denying themselves one of the most important advantages of using
Django and Python. If you are not aware of these tools, it’s well worth the time you spend
educating yourself.
As a Django (and Python) developer, make it your mission to use third-party libraries in-
stead of reinventing the wheel whenever possible. The best libraries have been written, doc-
umented, and tested by amazingly competent developers working around the world. Stand-
ing on the shoulders of these giants is the difference between amazing success and tragic
downfall.
As you use various packages, study and learn from their code. You’ll learn patterns and tricks
that will make you a better developer.
On the other hand, it’s very important to be able to identify the good packages from the
bad. It’s well worth taking the time to evaluate packages written by others the same way we
evaluate our own work. We cover this later in this chapter in Section 23.10: What Makes a
Good Django Package?
23.5 Tools for Installing and Managing Packages

To take full advantage of all the packages available for your projects, having virtualenv and
pip (or Conda or Poetry) installed isn’t something you can skip over. It’s mandatory.
Refer to Chapter 2: The Optimal Django Environment Setup for more details.
23.6 Package Requirements

As we mentioned earlier in Chapter 5: Settings and Requirements Files, we manage our
Django/Python dependencies with requirements files. These files go into the requirements/
directory that exists in the root of our projects.
23.7 Wiring Up Django Packages: The Basics

When you find a third-party package that you want to use, follow these steps:
Please submit
23.7.1 Step 1: Read the Documentation for the Package

Are you sure you want to use it? Make sure you know what you’re getting into before you
install any package.
23.7.2 Step 2: Add Package and Version Number to Your

Requirements
If you recall from Chapter 5: Settings and Requirements Files, a requirements/base.txt file
looks something like this (but probably longer):
Example 23.1: Adding Packages with Version Numbers to Requirements
Django==1.11
coverage==4.3.4
django-extensions==1.7.6
django-braces==1.11
Note that each package is pinned to a specific version number. Always pin your package
dependencies to version numbers.
What happens if you don’t pin your dependencies? You are almost guaranteed to run into
problems at some point when you try to reinstall or change your Django project. When new
versions of packages are released, you can’t expect them to be backwards-compatible.
Our sad example: Once we followed a software-as-a-service platform’s instructions for us-
ing their library. As they didn’t have their own Python client, but an early adopter had a
working implementation on GitHub, those instructions told us to put the following into
our requirements/base.txt:
Example 23.2: How Not To List Requirements
-e git+https://github.com/erly-adptr/py-junk.git#egg=py-jnk
Our mistake. We should have known better and pinned it to a particular git revision number.
Not the early adopter’s fault at all, but they pushed up a broken commit to their repo. Once
we had to fix a problem on a site very quickly, so we wrote a bug fix and tested it locally in
development. It passed the tests. Then we deployed it to production in a process that grabs
278 Please submit

23.8: Troubleshooting Third-Party Packages
all dependency changes; unfortunately the broken commit was interpreted as a valid change.
Which meant, while fixing one bug, we crashed the site.
Not a fun day.
The purpose of using pinned releases is to add a little formality and process to our published
work. Especially in Python, GitHub and other repos are a place for developers to pub-
lish their work-in-progress, not the final, stable work upon which our production-quality
projects depend.
One more thing, when pinning dependencies, try to pin the dependencies of dependencies.
It just makes deployment and testing that much more predictable.
23.7.3 Step 3: Install the Requirements Into Your Virtualenv

Assuming you are already in a working virtualenv and are at the <repo_root> of your
project, you pip install the appropriate requirements file for your setup, e.g. require-
ments/dev.txt.
If this is the first time you’ve done this for a particular virtualenv, it’s going to take a while
for it to grab all the dependencies and install them.
23.7.4 Step 4: Follow the Package’s Installation Instructions

Exactly
Resist the temptation to skip steps unless you’re very familiar with the package. Since open
source Django package developers tend to take pride in their documentation and love to get
people to use their packages, most of the time the installation instructions they’ve authored
make it easy to get things running.
23.8 Troubleshooting Third-Party Packages

Sometimes you run into problems setting up a package. What should you do?
First, make a serious effort to determine and solve the problem yourself. Pour over the
documentation and make sure you didn’t miss a step. Search online to see if others have run
into the same issue. Be willing to roll up your sleeves and look at the package source code,
as you may have found a bug.
If it appears to be a bug, see if someone has already reported it in the package repository’s
issue tracker. Sometimes you’ll find workarounds and fixes there. If it’s a bug that no one
has reported, go ahead and file it.
Please submit
If you still get stuck, try asking for help in all the usual places: StackOverflow, IRC #django,
the project’s IRC channel if it has its own one, and your local Python user group. Be as
descriptive and provide as much context as possible about your issue.
23.9 Releasing Your Own Django Packages

Whenever you write a particularly useful Django app, consider packaging it up for reuse in
other projects.
The best way to get started is to follow Django’s Advanced Tutorial: How to Write Reusable
Apps, for the basics: docs.djangoproject.com/en/3.2/intro/reusable-apps/
In addition to what is described in that tutorial, we recommend that you also:
ä Create a public repo containing the code. Most Django packages are hosted on
GitHub these days, so it’s easiest to attract contributors there, but various alterna-
tives exist (Gitlab, Bitbucket, Assembla, etc.).
ä Release the package on the Python Package Index (pypi.org). Follow
the submission instructions at packaging.python.org/distributing/
#uploading-your-project-to-pypi.
ä Add the package to Django Packages (djangopackages.org).
ä Use Read the Docs (readthedocs.io) or similar service to host your Sphinx docu-
mentation.
TIP: Where Should I Create a Public Repo?

There are websites that offer free source code hosting and version control for open
source projects. As mentioned in Chapter 2: The Optimal Django Environment
Setup, GitHub or GitLab are two popular options.
When choosing alternative hosted version control services, keep in mind that pip
only supports Git, Mercurial, Bazaar, and Subversion.
23.10 What Makes a Good Django Package?

Here’s a checklist for you to use when creating a new open source Django package. Much of
this applies to Python packages that are not Django-specific. This checklist is also helpful
for when you’re evaluating a Django package to use in any of your projects.
23.10.1 Purpose
Your package should do something useful and do it well. The name should be descriptive.
The package repo’s root folder should be prefixed with ‘django-’ or ‘dj-’ to help make it easier
to find.
280 Please submit

23.10: What Makes a Good Django Package?
If part of the package’s purpose can be accomplished with a related Python package that
doesn’t depend on Django, then create a separate Python package and use it as a dependency.
23.10.2 Scope
Your package’s scope should be tightly focused on one small task. This means that your
application logic will be tighter, and users will have an easier time patching or replacing the
package.
23.10.3 Documentation
A package without documentation is a pre-alpha package. Docstrings don’t suffice as docu-
mentation.
As described in Chapter 25: Documentation: Be Obsessed, your docs should be written in

Markdown. A nicely-formatted version of your docs should be generated with MkDocs,
Sphinx, or other tools and hosted publicly. We encourage you to use readthedocs.io
with webhooks so that your formatted documentation automatically updates whenever you
make a change.
If your package has dependencies, they should be documented. Your package’s installation
instructions should also be documented. The installation steps should be bulletproof.
23.10.4 Tests
Your package should have tests. Tests improve reliability, make it easier to advance
Python/Django versions, and make it easier for others to contribute effectively. Write up
instructions on how to run your package’s test suite. If you or any contributor can run your
tests easily before submitting a pull request, then you’re more likely to get better quality
contributions.
23.10.5 Templates
In the past, some Django packages provided instructions for creating templates in their docs
in lieu of actual template files. However, nowadays it’s pretty standard for Django packages
to come with a set of barebones templates that demonstrate basic functionality. Typically
these templates contain only minimalist HTML, any needed JavaScript, and no CSS. The
exception is for packages containing widgets that require CSS styling.
23.10.6 Activity
Your package should receive regular updates from you or contributors if/when needed.
When you update the code in your repo, you should consider uploading a minor or ma-
Please submit
jor release to the Python Package Index.
23.10.7 Community
Great open source packages, including those for Django, often end up receiving contribu-
tions from other developers in the open source community. All contributors should receive
attribution in a CONTRIBUTORS.rst or AUTHORS.rst file.
Be an active community leader if you have contributors or forks of your package. If your
package is forked by other developers, pay attention to their work. Consider if there are
ways that parts or all of their work can be merged into your fork. If the package’s function-
ality diverges a lot from your package’s purpose, be humble and consider asking the other
developer to give their fork a new name.
23.10.8 Modularity
Your package should be as easily pluggable into any Django project that doesn’t replace
core components (templates, ORM, etc) with alternatives. Installation should be minimally
invasive. Be careful not to confuse modularity with over-engineering, though.
23.10.9 Availability on PyPI

All major and minor releases of your package should be available for download from the
Python Package Index. Developers who wish to use your package should not have to go to
your repo to get a working version of it. Use proper version numbers per the next section.
23.10.10 Uses the Broadest Requirements Specifiers Possible

Your third-party package should specify in setup.py the install_requires argument
what other libraries your package requires in the broadest terms possible. However, this
is a terrible way to define a package’s requirements:
Example 23.3: Narrow Requirements for a Package
# DON'T DO THIS!
# requirements for django-blarg
Django==3.0
requests==2.11.0
The reason is dependency graphs. Every so often something that you absolutely pin to a
specific version of Django or another library will break on someone else’s site project. For
282 Please submit

example, what if icecreamratings.com was our site and this was its deployed project’s require-
ments.txt file, and we installed django-blarg?
Example 23.4: Requirements.txt for icecreamratings.com
# requirements.txt for the mythical web site 'icecreamratings.com'

Django==3.1
requests==2.13.0
django−blarg==1.0
# Note that unlike the django−blarg library , we explicitly pin

# the requirements so we have total control over the environment
What would happen if Bad Example 21.3 were installed to a project with Example 21.4
requirements is that the Django 3.0 requirement would overwrite the Django 3.1 specifica-
tion during installation of icecreamratings.com requirements. As there are several backwards
incompatibilities between Django 3.0 and 3.1, django-blarg could make icecreamratings.com
site simply throw HTTP 500 errors.
Your third-party package should specify what other libraries your package requires in the
broadest terms possible:
Example 23.5: Broadly Defined Package Requirements
# requirements for django-blarg
Django>=3.1,<3.0
requests>=2.13.0,<=3.0.0
Additional Reading:
ä pip.pypa.io/en/stable/reference/pip_install/
#requirement-specifiers
ä nvie.com/posts/pin-your-packages/
23.10.11 Proper Version Numbers

Like Django and Python, we prefer to adhere to the strict version of PEP 386 naming
schema. In fact we follow the ‘A.B.C’ pattern. Let’s go through each element:
Please submit
‘A’ represents the major version number. Increments should only happen with large changes
that break backwards compatibility from the previous major version. It’s not uncom-
mon to see large API changes between versions.
‘B’ is the minor version number. Increments include less breaking changes, or deprecation
notices about forthcoming changes.
‘C’ represents bug fix releases, and purists call this the ‘micro’ release. It’s not uncommon
for developers to wait until a project has its first release at this level before trying the
latest major or minor release of an existing project.
For alpha, beta, or release-candidates for a project, the convention is to place this informa-
tion as a suffix to the upcoming version number. So you might have:
ä Django 3.2-alpha
ä django-crispy-forms 1.9.1-beta
WARNING: Don’t Upload Unfinished Code to PyPI

PyPI, the Python Package Index, is meant to be the place where dependable, stable
packages can be harnessed to build Python projects. PyPI is not the place for Alpha,
Beta, or Release Candidate code, especially as pip and other tools will fetch the latest
release by default.
Be nice to other developers and follow the convention of only placing proper
releases on PyPI.
Note: While modern versions of pip no longer install pre-releases by default, it’s
dangerous to expect users of code to have the latest pip version installed.
Additional Reading:
ä python.org/dev/peps/pep-0386
ä semver.org
23.10.12 Name
The name of the project is absolutely critical. A well-named project makes it easy to discover
and remember, a poor name hides it from potential users, can scare off its use from some
developer shops, and even block it from being listed on PyPI, Django Packages, and other
resources.
284 Please submit

We did cover the basics in Section 4.2: What to Name Your Django Apps, but here are tips
that apply to open source Django packages:
ä Check to see that the name isn’t already registered on PyPI. Otherwise, it won’t be trivial
to install with pip.
ä Check to see that the name isn’t on Django Packages. This applies only to packages de-
signed for use with Django.
ä Don’t use names that include obscenity. While you might find them funny, it’s unfortu-
nate for others. For example a noted developer once created a library that couldn’t be
used at NASA until he agreed to change the name.
23.10.13 License
Your package needs a license. Preferably, for individuals it should be licensed under the MIT
licenses, which are generally accepted for being permissive enough for most commercial or
noncommercial uses. If you are worried about patents, then go with the Apache license.
Create a LICENSE.rst file in your repo root, mention the license name at the top, and paste
in the appropriate text from the (OSI) approved list at choosealicense.com.
TIP: Licenses Protect You and the World

In this era of casual litigation and patent trolls adding a software license isn’t just
a matter of protecting your ownership of the code. It’s much, much more. If you
don’t license your code, or use an unapproved license not vetted by real lawyers, you
run the risk of your work being used as a weapon by a patent troll, or in the case of
financial or medical disaster, you could be held liable.
OSI-approved licenses all include a couple critical statements on copyright, redis-
tribution, disclaimer of warranty, and limitation of liability.
23.10.14 Clarity of Code

The code in your Django package should be as clear and simple as possible. Don’t use weird,
unusual Python/Django hacks without explaining what you are doing.
23.10.15 Use URL Namespaces

Described in Section 8.4: Use URL Namespaces, URL namespaces allow for greater inter-
operability. Using means it’s easier to manage collisions between projects, or even prepare
for it ahead of time.
Please submit
If there is concern about future collisions, settings-based URL namespace systems can be
implemented. This is where the project defines its URL namespace as a setting, then pro-
vides a Django context processor and detailed instructions on use. While it’s not hard to
implement, it does create a level of abstraction that can make a project a little bit harder to
maintain.
23.11 Creating Your Own Packages the Easy Way

Releasing your own bit of code can be a wonderfully rewarding experience. Everyone should
do it!
That said, putting all the pieces together in order to make a reusable Django package is a lot
of work, and it’s common to get things wrong. Fortunately, Cookiecutter makes this easy.
PACKAGE TIP: Cookiecutter: Project Templates Made Easy

In 2013 Audrey created the popular Cookiecutter utility for generating project tem-
plates. It’s easy to use and very powerful. Numerous templates exist for Python and
Django packages. Even better, many IDEs such as PyCharm and Visual Studio
Code now provide support for Cookiecutter-based templates.
ä github.com/cookiecutter/cookiecutter
ä cookiecutter.readthedocs.io
In the Cookiecutter templates referenced below, we have vetted them by aggressively asking
for them to be reviewed by leaders in both the Django and Python communities. Just use
the following bash example at the command-line:
Example 23.6: Using Cookiecutter to Jumpstart Packages
# Only if you haven't installed cookiecutter yet

\$ pip install cookiecutter
# Creating a Django Package from scratch

\$ cookiecutter
,→ https://github.com/pydanny/cookiecutter-djangopackage.git
# Creating a Python Package from scratch

\$ cookiecutter
,→ https://github.com//ionelmc/cookiecutter-pylibrary.git
You’ll be prompted to provide information. The generated result will be an implementation

of a base Django/Python/etc. package template that includes code, documentation, tests,
286 Please submit

23.12: Maintaining Your Open Source Package
license, and much more.
23.12 Maintaining Your Open Source Package

WARNING: Open Source Burnout and Giving Too Much
Unless you are getting paid professionally to do open source work, remember that
this is volunteer work done for pleasure. Do what you can at your own pace, and
just try your best.
The open source packages that you create have a life of their own. They mature over time,
changing as their needs and the development standards grow over time. Here are some
things we should do when maintaining open source projects:
23.12.1 Give Credit for Pull Requests

When someone submits a pull request that’s accepted, treat them right. Make sure to add
the contributor to a project’s author document called something like CONTRIBUTORS.md
or AUTHORS.md.
23.12.2 Handling Bad Pull Requests

Sometimes you get pull requests that you just have to reject. Be nice and positive about it,
since a well-handled rejected pull request can make a friend for life.
Here are problematic pull requests that should be considered for rejection:
ä Any pull request that fails the tests. Ask for fixes. See Chapter 24: Testing Stinks and
Is a Waste of Money!.
ä Any added code that reduces test coverage. Again, see chapter 24.
ä Pull requests should change/fix as little as possible. Large, wide-sweeping changes in a
pull request should be rejected, with comments to isolate changes in smaller, atomic
pull requests.
ä Overly complex code submissions should be carefully considered. There is nothing wrong
with asking for simplification, better comments, or even rejecting an overly complex
pull request.
ä Code that breaks PEP-8 needs to be resubmitted. The Django world follows PEP-8 very
closely, and so should your project. Submissions that violate PEP 8 can be requested
to be improved.
ä Code changes combined with major whitespace cleanup. If someone submits a change of
two lines of code and corrects 200 lines of whitespace issues, the diff on that pull
Please submit
request is functionally unreadable and should be rejected. Whitespace cleanups need

to be in their own pull request.
WARNING: Code Changes Combined With Major Whitespace

Cleanup
We’re adding a warning because this is arguably a form of code obfuscation by a
third party. One could argue it’s potentially a security risk. What better way to inject
malignant code than through a pull request?
23.12.3 Do Formal PyPI Releases

In the Python community, it’s considered irresponsible to force developers to rely on a ‘stable’
master or trunk branch of critical open source projects because the PyPI version is out of
date. This can cause problems as open source code repositories are not considered to be good
sources of production quality code. For example, which particular commit or tag should be
used? On the other hand, PyPI, is a known resource designed to securely provide valid
installable packages.
In the Python world, the accepted best practice is to release when significant (or even minor)
changes or bug fixes happen on trunk or master. In fact, minor bug fix releases are a part of
every ongoing software project and no one faults anyone for these kinds of things (except
in US government IT contracts, but that’s outside the scope of this book).
If you aren’t sure how this works, please look at python-request’s change history, it be-
ing one of Python’s most popular projects: github.com/psf/requests/blob/master/
HISTORY.md
To create and upload your distribution, use the following steps:
Example 23.7: Using Twine to Upload Package Distributions
pip install twine

python setup.py sdist
twine upload dist/*
PACKAGE TIP: What is Twine?

Twine is the preferred library for uploading packages to PyPI. The problem with
python setup.py is that it sends files over a non secure connection, exposing
288 Please submit

23.12: Maintaining Your Open Source Package
your library to a man-in-the-middle attack. In contrast, twine uses only verified

TLS to upload your package.
That’s not all! Twine works better at uploading Wheels (see the next subsection),
doesn’t require executing the setup.py, and even pre-signs your releases. If you are
seriously security minded, it’s the tool of choice.
23.12.4 Create and Deploy Wheels to PyPI

According to PEP 427, Wheels are the new standard of python distribution. They are in-
tended to replace eggs and provide a number of advantages including faster installation and
allow secure digital signing. Support is offered in pip >= 1.4 and setuptools >= 0.8.
Example 23.8: Installing Wheel
pip install wheel
Then, after you’ve deployed your package to PyPI, run the following commands:
Example 23.9: Creating Wheel Distributions and Uploading Them
python setup.py bdist_wheel

twine upload dist/*
Twine makes universal wheels when the optional setup.cfg file is at the same level as setup.py
and includes this snippet:
Example 23.10: Configuring Universal Wheels
# setup.cfg
[wheel]
universal = 1
Wheel Resources:
Specification: PEP 427 python.org/dev/peps/pep-0427

Wheel Package on PyPI https://feld.to/3eTOdyS/project/wheel
Documentation wheel.readthedocs.io
Advocacy pythonwheels.com
Please submit
23.12.5 Add Git Tags to the Repo

We also like to git tag our releases in the repo. This is a snapshot of the code at the time the
release was submitted to PyPI (see previous subsection). In addition to serving as a historical
record, it can also location for installing packages from. For example, if an organization
doesn’t want to stand up a private package server, they can rely on git tags (or releases for
those repo hosts like GitHub that provide this option).
Example 23.11: Create and Push Tag to repo
git tag -a v1.4 -m "my version 1.4"

git push origin v1.4
Reference: git-scm.com/book/en/v2/Git-Basics-Tagging
23.12.6 Upgrade the Package to New Versions of Django

Every once in awhile, Django is updated with a minor release. Approximately once a year
there is a major Django release. When this happens, it’s very important to run our package’s
test suite in a virtualenv that contain Django’s latest release.
If for no other reason, this is an excellent reason to include tests in your project.
23.12.7 Follow Good Security Practices

We discuss security in-depth in Chapter 28: Security Best Practices. However, core Django,
Python, and PyPy developer Alex Gaynor has an incredibly useful article for maintainers
of any open source project:
alexgaynor.net/2013/oct/19/security-process-open-source-projects
TIP: Alex Gaynor on Security for Open Source Projects

“Security vulnerabilities put your users, and often, in turn, their users at
risk. As an author and distributor of software, you have a responsibility
to your users to handle security releases in a way most likely to help
them avoid being exploited.”
23.12.8 Provide Sample Base Templates

Always include some basic templates for views using your project. We prefer to write either
incredibly simple HTML or use a common front-end frameworks such as Bootstrap or Tail-
wind. This makes ‘test-driving’ the project much easier for developers who are considering
290 Please submit

23.13: Additional Reading
using it to solve their problems. Invariably they’ll modify the templates inside their own
templates/ directory, but this just makes everything so much easier.
In addition, include a templates/myapp/base.html to increase interoperability. You can see a

description and example of this in cookiecutter-djangopackage:
bit.ly/2onSzCV
23.12.9 Give the Package Away

Sometimes, life takes you away from maintaining a package. It might be family or a new
job, but sometimes you just have no need for a particular open source project. Time consid-
erations might mean that you don’t have the ability to review pull requests or explore ideas
for new features. If you’re the creator of a project it can be extremely challenging to let it go.
However, by giving a project away to active maintainers, it can be reborn and prove more
useful. It also earns the respect of the developer community at large.
Some notable giveaways in the Django and Python communities include:
ä Ian Bicking and pip/virtualenv.

ä Daniel and Audrey Feldroy and djangopackages.org
ä Daniel Feldroy and django-uni-form, dj-stripe, and django-mongonaut
ä Audrey Feldroy and Cookiecutter
ä Rob Hudson and django-debug-toolbar.
23.13 Additional Reading

The following are links to useful articles for anyone contributing to, creating, or maintaining
open source libraries:
ä djangoappschecklist.com A personal favorite of ours, the Django Apps Check-

list is a checklist for everything in this chapter.
ä alexgaynor.net/2013/sep/26/effective-code-review
ä hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty
23.14 Summary
Django’s real power is in the vast selection of third-party packages available to you for use
in your Django projects.
Make sure that you have pip and virtualenv (or Poetry as an alternative) installed and know
how to use them, since they’re your best tools for installing packages on your system in a
manageable way.
Please submit
Get to know the packages that exist. The Python Package Index and Django Packages are
a great starting point for finding information about packages.
Package maturity, documentation, tests, and code quality are good starting criteria when
evaluating a Django package.
Installation of stable packages is the foundation of Django projects big and small. Being
able to use packages means sticking to specific releases, not just the trunk or master of a
project. Barring a specific release, you can rely on a particular commit. Fixing problems that
a package has with your project takes diligence and time, but remember to ask for help if
you get stuck.
We also covered how to create your own third-party package, and provided basic instruction
on how to use cookiecutter to jump-start you on your way to releasing something on the
Python Package Index. We also included instructions on using the new Wheel format.
Finally, we provided guidance on how to maintain a package.
292 Please submit

24 | Testing Stinks and Is a Waste of
Money!
There, got you to this chapter.
Now you have to read it.
We’ll try and make this chapter interesting.
24.1 Testing Saves Money, Jobs, and Lives

Daniel’s Story: Ever hear the term “smoke test”?
Gretchen Davidian, a Management and Program Analyst at NASA, told me that when she
was still an engineer, her job as a tester was to put equipment intended to get into space
through such rigorous conditions that they would begin emitting smoke and eventually
catch on fire.
That sounds exciting! Employment, money, and lives were on the line, and knowing
Gretchen’s attention to detail, I’m sure she set a lot of hardware on fire.
Keep in mind that for a lot of us, as software engineers, the same risks are on the line
as NASA. I recall in 2004 while working for a private company how a single miles-vs-
kilometers mistake cost a company hundreds of thousands of dollars in a matter of hours.
Quality Assurance (QA) staff lost their jobs, which meant that money and health benefits
were gone. In other words, employment, money, and possibly lives can be lost without ad-
equate tests. While the QA staff were very dedicated, everything was done via manually
clicking through projects, and human error simply crept into the testing process.
Today, as Django moves into a wider and wider set of applications, the need for automated
testing is just as important as it was for Gretchen at NASA and for the poor QA staff in 2004.
Here are some cases where Django is used today that have similar quality requirements:
Please submit
Chapter 24: Testing Stinks and Is a Waste of Money!
ä Applications that handle medical information.

ä Applications that provide life-critical resources to people in need.
ä Applications working with user’s financial information.
PACKAGE TIP: Useful Library for Testing Django Projects

We like to use coverage.py.
This tool provides clear insight into what parts of your code base are covered by tests,
and what lines haven’t been touched by tests. You also get a handy percentage of how
much of your code is covered by tests. Even 100% test coverage doesn’t guarantee a
bug-free application, but it helps.
We want to thank Ned Batchelder for his incredible work in maintaining cover-
age.py. It’s a superb project and is useful for any Python-related project.
24.2 How to Structure Tests

Let’s say we’ve just created a new Django app. The first thing we do is delete the default but
useless tests.py module that python manage.py startapp creates. In it’s place we create
a directory called tests and place an empty __init__.py within.
Inside that new directory, because most apps need them, we create test_forms.py,
test_models.py, test_views.py modules. Tests that apply to forms go into test_forms.py, model
tests go into test_models.py, and so on.
Here’s what it looks like:
Example 24.1: How to Structure Tests
popsicles/
__init__.py
admin.py
forms.py
models.py
tests/
__init__.py
test_forms.py
test_models.py
test_views.py
views.py
294 Please submit

24.3: How to Write Unit Tests
Also, if we have other files besides forms.py, models.py and views.py that need testing, we
create corresponding test files.
We like this approach because while it does add an extra layer of nesting, the alternative is
to have an app structure with a painful number of modules to navigate.
TIP: Prefix Test Modules With test_

It’s critically important that we always prefix test modules with test_, otherwise
Django’s test runner can’t trivially discover our test files. Also, it’s a nice convention
that allows for greater flexibility for viewing filenames in IDEs and text editors. As
always, never lean on your preferred IDE for how to name things.
24.3 How to Write Unit Tests

It’s not uncommon for programmers to feel at the top of their game at the moment they are
writing code. When they revisit that same code in months, weeks, days, or even hours and
it’s not uncommon for programmers to feel as if that same code is of poor quality.
The same applies to writing unit tests.
Over the years, we’ve evolved a number of practices we like to follow when writing tests,
including unit tests. Our goal is always to write the most meaningful tests in the shortest
amount of time. Hence the following:
24.3.1 Each Test Method Tests One Thing

A test method must be extremely narrow in what it tests. A single unit test should never
assert the behavior of multiple views, models, forms, or even multiple methods within a
class. Instead, a single test should assert the behavior of a single view, model, form, method,
or function.
Therein lies a conundrum. How does one run a test for a view, when views often require the
use of models, forms, methods, and functions?
The trick is to be absolutely minimalistic when constructing the environment for a particular
test, as shown in the example below:
Example 24.2: Testing Just One Thing
# flavors/tests/test_api.py
import json
Please submit
from django.test import TestCase

from django.urls import reverse
class FlavorAPITests(TestCase):
def setUp(self):
Flavor.objects.get_or_create(title='A Title',
,→ slug='a-slug')
def test_list(self):
url = reverse('flavors:flavor_object_api')
response = self.client.get(url)
self.assertEquals(response.status_code, 200)
data = json.loads(response.content)
self.assertEquals(len(data), 1)
In this test, taken from code testing the API we presented in Section 17.2: Illustrating
Design Concepts With a Simple API, we use the setUp() method to create the minimum
possible number of records needed to run the test.
Here’s a much larger example, one based on the REST API example that we provided in
Chapter 17: Building REST APIs With Django REST Framework.
Example 24.3: Testing API Code
# flavors/tests/test_api.py
import json
from django.test import TestCase

from django.urls import reverse
class DjangoRestFrameworkTests(TestCase):
def setUp(self):
Flavor.objects.get_or_create(title='title1', slug='slug1')
296 Please submit

Flavor.objects.get_or_create(title='title2', slug='slug2')
self.create_read_url = reverse('flavors:flavor_rest_api')
self.read_update_delete_url = \
reverse('flavors:flavor_rest_api', kwargs={'slug':
,→ 'slug1'})
def test_list(self):
response = self.client.get(self.create_read_url)
# Are both titles in the content?

self.assertContains(response, 'title1')
self.assertContains(response, 'title2')
def test_detail(self):
response = self.client.get(self.read_update_delete_url)
content = {'id': 1, 'title': 'title1', 'slug': 'slug1',
'scoops_remaining': 0}
self.assertEquals(data, content)
def test_create(self):
post = {'title': 'title3', 'slug': 'slug3'}
response = self.client.post(self.create_read_url, post)
content = {'id': 3, 'title': 'title3', 'slug': 'slug3',
'scoops_remaining': 0}
self.assertEquals(data, content)
self.assertEquals(Flavor.objects.count(), 3)
def test_delete(self):
response = self.client.delete(self.read_update_delete_url)
self.assertEquals(Flavor.objects.count(), 1)
24.3.2 For Views, When Possible Use the Request Factory

The django.test.client.RequestFactory provides a way to generate a request in-
stance that can be used as the first argument to any view. This provides a greater amount of
isolation than the standard Django test client, but it does require a little bit of extra work on
Please submit
the part of the test writer. This is because the request factory doesn’t support middleware,
including session and authentication.
See docs.djangoproject.com/en/3.2/topics/testing/advanced/
Unfortunately the documentation doesn’t cover when you want to test a view wrapped with
a single middleware class. For example, if a view required sessions, this is how we would
do it:
Example 24.4: How to Add Middleware to Requests and Responses
from django.contrib.auth.models import AnonymousUser

from django.contrib.sessions.middleware import SessionMiddleware
from django.test import TestCase, RequestFactory
from .views import cheese_flavors
def add_middleware_to_request(request, middleware_class):

middleware = middleware_class()
middleware.process_request(request)
return request
def add_middleware_to_response(request, middleware_class):

middleware = middleware_class()
middleware.process_response(request)
return request
class SavoryIceCreamTest(TestCase):
def setUp(self):
# Every test needs access to the request factory.
self.factory = RequestFactory()
def test_cheese_flavors(self):
request = self.factory.get('/cheesy/broccoli/')
request.user = AnonymousUser()
# Annotate the request object with a session

request = add_middleware_to_request(request,
,→ SessionMiddleware)
request.session.save()
# process and test the request
298 Please submit

response = cheese_flavors(request)
self.assertContains(response, 'bleah!')
24.3.3 Don’t Write Tests That Have to Be Tested

Tests should be written as simply as possible. If the code in a test (or the code called to help
run a test) feels complicated or abstracted, then we have a problem. In fact, we ourselves
are guilty of writing overly complicated utility test functions that required their own tests
in the past. As one can imagine, this made debugging the actual tests a nightmare.
24.3.4 Don’t Repeat Yourself Doesn’t Apply to Writing Tests

The setUp() method is really useful for generating reusable data across all test methods in
a test class. However, sometimes we need similar but different data between test methods,
which is where we often fall into the trap of writing fancy test utilities. Or worse, we decide
that rather than write 20 similar tests, we can write a single method that when passed certain
arguments will handle all the work for us.
Our favorite method of handling these actions is to just dig in and write the same or sim-
ilar code multiple times. In fact, we’ll quietly admit to copy/pasting code between tests to
expedite our work.
TIP: Our Problem With the Django Testing Tutorial #5

The official Django beginner tutorial on testing demonstrates the implementation
of a create_question() utility method, which is designed to take some of the
repetition out of the process of created questions.
We think its inclusion in the tutorial is a mistake. Simple examples in the official
documentation that follow poor practice provide encouragement that often play
out in terrible ways. We’ve encountered projects inspired by this example to add
deep layers of abstraction to their testing code, abstraction that makes it incredibly
challenging to correct and enhance existing tests.
Again, Don’t Repeat Yourself doesn’t apply to tests.
24.3.5 Don’t Rely on Fixtures

We’ve learned over time that using fixtures is problematic. The problem is that fixtures are
hard to maintain as a project’s data changes over time. Modifying JSON-formatted files to
match your last migration is hard, especially as it can be difficult to identify during the JSON
Please submit
load process where your JSON file(s) is either broken or a subtly inaccurate representation
of the database.
Rather than wrestle with fixtures, we’ve found it’s easier to write code that relies on the
ORM. Other people like to use third-party packages.
PACKAGE TIP: Tools to Generate Test Data

The following are popular tools for test data generation:
ä factory boy A package that generates model test data.
ä faker This package generates test data, but rather than a random jumble of
text, it creates localized names, addresses, and text. It even comes with instruc-
tions on how to integrate it with factory boy: faker.readthedocs.io/en/
master/
#how-to-use-with-factory-boy
ä model bakery Another package that generates model test data.
ä mock Not explicitly for Django, this allows us to replace parts of our system
with mock objects. This project made its way into the standard library as of
Python 3.3.
24.3.6 Things That Should Be Tested

Everything! Seriously, test whatever is possible to test, including:
Views: Viewing of data, changing of data, and custom class-based view methods.
Models: Creating/updating/deleting of models, model methods, model manager methods.
Forms: Form methods, clean() methods, and custom fields.
Validators: Really dig in and write multiple test methods against each custom validator
you write. Pretend you are a malignant intruder attempting to damage the data in the
site.
Signals: Since they act at a distance, signals can cause grief especially if you lack tests on
them.
Filters: Since filters are essentially just functions accepting one or two arguments, writing
tests for them should be easy.
Template Tags: Since template tags can do anything and can even accept template context,
writing tests often become much more challenging. This means you really need to test
them, since otherwise you may run into edge cases.
Miscellany: Context processors, middleware, email, and anything else not covered in this
list.
300 Please submit

Failure What happens when any of the above fail? Testing for system error is as important
as testing for system success.
The only things that shouldn’t be tested are parts of your project that are already covered by
tests in core Django and third-party packages. For example, a model’s fields don’t have to be
tested if you’re using Django’s standard fields as-is. However, if you’re creating a new type
of field (e.g. by subclassing FileField), then you should write detailed tests for anything
that could go wrong with your new field type.
Figure 24.1: Test as much of your project as you can, as if it were free ice cream.
24.3.7 Test for Failure

Let’s say we have a view that allows users to edit their own ice cream shop reviews. Typical
tests involve logging in, attempting to change the review, and then checking whether they’ve
actually changed. Test success, coverage 100%. Right?
However, this only tests part of the scenario. What if the user isn’t logged in? What if
the user is trying to edit someone else’s review? Does the view produce an error and most
importantly: is the object left unchanged? It has been argued that this test is even more
important than the success scenario: a failure in the success scenario will cause inconvenience
for users but will be reported. A failure in the fail scenario will cause a silent security hole
that could go undetected until it’s too late.
This is only a sampling of the things that can go wrong when we don’t test for what happens
when our systems break down. It is up to us to learn how to test for the exceptions our code
Please submit
may throw:
ä docs.python.org/2/library/unittest.html#unittest.TestCase.
assertRaises
ä bit.ly/2pAxLtm PyTest assertion docs
24.3.8 Use Mock to Keep Unit Tests From Touching the World
Unit tests are not supposed to test things external to the function or method they are call-
ing. Which means that during tests we should not access external APIs, receive emails or
webhooks, or anything that is not part of the tested action. Alas, this causes a conundrum
when you are trying to write a unit test for a function that interacts with an external API.
At this point you have two choices:
ä Choice #1: Change the unit test to be an Integration Test.

ä Choice #2: Use the Mock library to fake the response from the external API.
The Mock library, created by Michael Foord, has as one of its features the capability to
briefly monkey-patch libraries in order to make them return the exact value that we want.
This way we are testing not the availability of the external API, but instead just the logic of
our code.
In the example displayed below, we are monkey-patching a function in a mythical Ice Cream
API library so our test code doesn’t access anything external to our application.
Example 24.5: Using Mock to Keep Unit Tests From Touching the World
from unittest import mock, TestCase
import icecreamapi
from flavors.exceptions import CantListFlavors

from flavors.utils import list_flavors_sorted
class TestIceCreamSorting(TestCase):
# Set up monkeypatch of icecreamapi.get_flavors()

@mock.patch.object(icecreamapi, 'get_flavors')
def test_flavor_sort(self, get_flavors):
# Instructs icecreamapi.get_flavors() to return an
,→ unordered list.
302 Please submit

get_flavors.return_value = ['chocolate', 'vanilla',

,→ 'strawberry', ]
# list_flavors_sorted() calls the icecreamapi.get_flavors()

# function. Since we've monkeypatched the function, it
,→ will always
# return ['chocolate', 'strawberry', 'vanilla', ]. Which
,→ the.
# list_flavors_sorted() will sort alphabetically
flavors = list_flavors_sorted()
self.assertEqual(
flavors,
['chocolate', 'strawberry', 'vanilla', ]
Now let’s demonstrate how to test the behavior of the list_flavors_sorted() function
when the Ice Cream API is inaccessible.
Example 24.6: Testing For When API is Unavailable
@mock.patch.object(icecreamapi, 'get_flavors')
def test_flavor_sort_failure(self, get_flavors):
# Instructs icecreamapi.get_flavors() to throw a FlavorError.
get_flavors.side_effect = icecreamapi.FlavorError()
# list_flavors_sorted() catches the icecreamapi.FlavorError()

# and passes on a CantListFlavors exception.
with self.assertRaises(CantListFlavors):
list_flavors_sorted()
As an added bonus for API authors, here’s how we test how code handles two different
python-requests connection problems:
Example 24.7: Testing python-requests Connection Failures
@mock.patch.object(requests, 'get')
def test_request_failure(self, get):
"""Test if the target site is inaccessible."""
Please submit
get.side_effect = requests.exception.ConnectionError()
@mock.patch.object(requests, 'get')
def test_request_failure_ssl(self, get):
"""Test if we can handle SSL problems elegantly."""
get.side_effect = requests.exception.SSLError()
24.3.9 Use Fancier Assertion Methods

Comparing two lists (or tuples) is a very common use case. However, if the lists are al-
lowed to have different sort orders, then we have to sort the lists to match, then run
self.assertEqual(control_list, candidate_list) right?
Not if we know to use unittest’s assertItemsEqual() assertion method! In fact, Python

and Django’s unittest documentation includes handy links to the very useful assertion types
we get for free:
ä docs.python.org/3/library/unittest.html#assert-methods
ä docs.djangoproject.com/en/3.2/topics/testing/tools/#assertions
We’ve found the following assert methods extremely useful:
ä assertRaises
ä assertRaisesMessage()
ä assertCountEqual()
ä assertDictEqual()
ä assertFormError()
ä assertContains() Check status 200, checks in response.content.
ä assertHTMLEqual() Amongst many things, ignores whitespace differences.
ä assertInHTML() Great for confirming a tiny bit of HTML in a giant HTML page.
ä assertJSONEqual()
ä assertURLEqual()
304 Please submit

24.4: What About Integration Tests?
24.3.10 Document the Purpose of Each Test

Just as it is a good idea to document the purpose of a class, method, or function with doc-
strings, it is also a good idea to document the purpose of the test analogs of these items.
If undocumented code makes a project somewhat harder to maintain, undocumented test
code can make a project impossible to test. To remedy this, a little bit of docstring can go a
long way.
If one thinks this is boring, well, we’ve found that a good way to deal with an impossible-
to-debug problem is to document the related tests. By the time the tests are documented,
we have either figured out the problem or we have documented tests. Either case is a win!
Resources:
ä hynek.me/articles/document-your-tests/
ä interrogate.readthedocs.io/ - A library that checks code bases for missing
docstrings
24.4 What About Integration Tests?

Integration testing is when individual software modules are combined and tested as a group.
This is best done after unit tests are complete. Examples of integration tests can include:
ä Selenium tests to confirm that an application works in the browser.

ä Actual testing against a third-party API instead of mocking responses. For example,
Django Packages conducts periodic tests against GitHub and the PyPI API to ensure
that its interaction with those systems is valid.
ä Interacting with httpbin.org to confirm the validity of outbound requests.
ä Using runscope.com or postman.com to validate that our API is working as ex-
pected.
Integration tests are a great way to confirm that ‘all the pieces’ are working. We can confirm
that our users will see what they are supposed to see and our APIs are functioning correctly.
The downside of integration tests are:
ä Setting up integration tests can take a lot of time.

ä Compared to unit tests, integrations are extremely slow. That’s because instead of
testing the smallest components, integration tests are, by definition, testing the whole
system.
ä When errors are thrown by integration tests, uncovering the problem is harder than
unit tests. For example, a problem affecting a single type of browser might be caused
by a unicode transformation happening at the database level.
Please submit
ä Integration tests are fragile compared to unit tests. A small change in a component
or setting can break them. We’ve yet to work on a significant project where at least
one person wasn’t forever blocked from running them successfully.
Even with these problems, integration tests are useful and worth considering adding to our
testing stack.
24.5 Continuous Integration

For projects of any size, we recommend setting up a continuous integration (CI) server to
run the project’s test suite whenever code is committed and pushed to the project repo. See
Chapter 34: Continuous Integration for more details.
24.6 Who Cares? We Don’t Have Time for Tests!

“Tests are the Programmer’s stone, transmuting fear into boredom.”
–Kent Beck
Let’s say you are confident of your coding skill and decide to skip testing to increase your
speed of development. Or maybe you feel lazy. It’s easy to argue that even with test genera-
tors and using tests instead of the shell, they can increase the time to get stuff done.
Oh, really?
What about when it’s time to upgrade?
That’s when the small amount of work you did up front to add tests saves you a lot of work.
For example, in the summer of 2010, Django 1.2 was the standard when we started Django
Packages (djangopackages.org). Since then the project has stayed current with new
Django versions, which has been really useful. Because of its pretty good test coverage,
moving it up a version of Django (or the various dependencies) has been easy. Our path to
upgrade:
ä Upgrade the version in a local instance of Django Packages.

ä Run the tests.
ä Fix any errors that are thrown by the tests.
ä Do some manual checking.
If Django Packages didn’t have tests, any time we upgraded anything we would have to click
through dozens and dozens of scenarios manually, which is error-prone. Having tests means
we can make changes and dependency upgrades with the confidence that our users (i.e. the
Django community) won’t have to deal with a buggy experience.
This is the benefit of having tests.
306 Please submit

24.7: The Game of Test Coverage
24.7 The Game of Test Coverage

A great, fun game to play is trying get test coverage as high as possible. Every day that we
increase our test coverage is a victory, and every day that the coverage goes down is a loss.
24.8 Setting Up the Test Coverage Game

Yes, we call test coverage a game. It’s a good tool for developers to push themselves. It’s
also a nice metric that both developers and their clients/employers/investors can use to help
evaluate the status of a project.
We advocate following these steps because most of the time we want to only test our own
project’s apps, not all of Django, and the myriad of third-party libraries that are the building
blocks of our project. Testing those ‘building blocks’ takes an enormous amount of time,
which is a waste because most are already tested or require additional setup of resources.
24.8.1 Step 1: Start Writing Tests

We’ve done that already, right?
24.8.2 Step 2: Run Tests and Generate Coverage Report

Let’s try it out! In the command-line, at the <project_root>, type:
Example 24.8: Running Django tests using coverage.py
$ coverage run manage.py test --settings=twoscoops.settings.test
If we have nothing except for the default tests for two apps, we should get a response that
looks like:
Example 24.9: Positive Test Results
Creating test database for alias "default"...

..
-----------------------------------------------
Ran 2 tests in 0.008s
OK
Destroying test database for alias "default"...
Please submit
This doesn’t look like much, but what it means is that we’ve constrained our application to
only run the tests that you want. Now it’s time to go and look at and analyze our embarrass-
ingly low test coverage numbers.
24.8.3 Step 3: Generate the Report!

coverage.py provides a very useful method for generating HTML reports that don’t just
provide percentage numbers of what’s been covered by tests, it also shows us the places
where code is not tested. In the command-line, at the <project_root>:
Example 24.10: Test Results Without admin.py
$ coverage html --omit="admin.py"
Ahem...don’t forget to change <project-root> to match the development machine’s structure!

For example, depending on where one does things, the <path-to-project-root> could be:
ä /Users/audreyr/code/twoscoops/twoscoops/
ä /Users/pydanny/projects/twoscoops/twoscoops/
ä c:\ twoscoops
After this runs, in the <project_root> directory there is a new directory called htmlcov/ . In
the htmlcov/ directory, open the index.html file using any browser.
What is seen in the browser is the test results for our test run. Unless we already wrote some
tests, the total on the front page will be in the single digits, if not at 0%. Click into the
various modules listed and we should see lots of code that’s red-colored. Red is bad.
Let’s go ahead and admit that our project has a low coverage total. If your project has a
low coverage total, you need to admit it as well. It’s okay just so long as we also resolve to
improve the coverage total.
In fact, there is nothing wrong in saying publicly that you are working to improve a project’s
test coverage. Then, other developers (including ourselves) will cheer you on!
24.9 Playing the Game of Test Coverage

The game has a single rule:
Mandate that no commit can lower test coverage.
308 Please submit

24.10: Alternatives to unittest
If we add a feature or bug fix and coverage is 65% when we start, we can’t merge our code
in until coverage is at least 65% again. At the end of each day, if test coverage goes up by
any amount, it means we’re winning.
Keep in mind that the gradual increase of test coverage can be a very good thing over huge
jumps. Gradual increases can mean that we developers aren’t putting in bogus tests to bump
up coverage numbers; instead, we are improving the quality of the project.
24.10 Alternatives to unittest

All the examples in this chapter thus far have used the unittest library. While every known
authority on testing agrees that unittest is a very powerful, useful tool, not all of them like it.
The specific reason, and one we fully comprehend, is that it requires too much boilerplate.
Fortunately, there is the pytest alternatives that requires a lot less boilerplate: pypi.org/
project/pytest-django/
This library is a wrapper around the pytest library. In return for a little bit of extra setup,
this allows for not just the running of unittest-based tests, but also for running any function
(and class/directory/module) prefixed with “test_”. For example, you could write a simple
test that looks like this:
Example 24.11: py.test example
# test_models.py
from pytest import raises
from cones.models import Cone
def test_good_choice():
assert Cone.objects.filter(type='sugar').count() == 1
def test_bad_cone_choice():
with raises(Cone.DoesNotExist):
Cone.objects.get(type='spaghetti')
While this example is based on pytest, similar functionality can be used with nose and it’s
nose.tools.raises decorator.
A possible downside of the simplicity of these function-based tests is the lack of inheritance.
If a project needs to have similar behavior over a lot of different tests, then writing tests this
way may not make sense.
Please submit
24.11 Summary
All of this might seem silly, but testing can be very serious business. In a lot of developer
groups this subject, while gamified, is taken very seriously. Lack of stability in a project can
mean the loss of clients, contracts, and even employment.
In the next chapter we cover a common obsession of Python developers: documentation.
310 Please submit

25 | Documentation: Be Obsessed
Given a choice between ice cream and writing great documentation, most Python developers
would probably choose to write the documentation. That being said, writing documentation
while eating ice cream is even better.
When you have great documentation tools like Markdown , MkDocs, Sphinx, Myst-
Parser for Sphinx, and you actually can’t help but want to add docs to your projects.
25.1 Use GitHub-Flavored Markdown for Docs

You’ll want to learn and follow the standard best practices for documentation. These days,
GitHub-Flavored Markdown (GFM) is the most common markup language used for doc-
umenting Python projects. Other tools and companies have adopted GFM, for example,
GitLab uses GFM in their READMEs and issue trackers.
Here is the formal GFM specification and a sample project which benefits from using it:
ä github.github.com/gfm/
ä django-rest-framework.org/
While it’s possible to study the formal documentation for GFM and learn the basics, here
is a quick primer of some very useful commands you should learn.
Example 25.1: Markdown Primer
# H1 Header
**emphasis (bold/strong)**
*italics*
_underline_
Please submit
Chapter 25: Documentation: Be Obsessed
link: [Two Scoops Press](https://twoscoopspress.com)
Note: The secret to links in Markdown is think of it like a

,→ function call. The link in the paranthesis is the value being
,→ called.
## Section Header
#. An enumerated list item

#. Second item
- First bullet
- Second bullet
- Indented Bullet
- Note carriage return and indents
Literal code block:
``` python
def like():
print("I like Ice Cream")
for i in range(10):
like()
```
JavaScript colored code block:
``` js
console.log("Don't use alert()");
```
312 Please submit


Two Scoops of Django 3.x by Daniel Audrey Feldroy (251-350)

Uploaded by

Copyright:

Available Formats

Two Scoops of Django 3.x by Daniel Audrey Feldroy (251-350)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Two Scoops of Django 3.x by Daniel Audrey Feldroy (251-350)

Uploaded by

Copyright:

Available Formats

17.

2: Illustrating Design Concepts With a Simple API

TIP: Classy Django REST Framework Is a Useful Reference

Now we’ll wire this into our flavors/urls.py module:

Example 17.5: Wiring in API Views

from flavors.api import views

Url View Url Name (same)

Table 17.3: URLConf for the Flavor REST APIs

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 213

WARNING: Our Simple API Does Not Use Permissions

The end result is the traditional REST-style API definition:

Example 17.6: Traditional REST-style API definition

TIP: Common Syntax for Describing REST APIs

17.3 REST API Architecture

17.3.1 Use Consistent API Module Naming

Example 17.7: Our Preferences for Naming API-Related Modules

214 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

Prepared exclusively for LAKLAK LAKLAK ([email protected]) Transaction: 11322

│ ├── __init__.py

Please observe the following:

17.3.2 Code for a Project Should Be Neatly Organized

17.3.3 Code for an App Should Remain in the App

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 215

17.3.4 Try to Keep Business Logic Out of API Views

216 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

Prepared exclusively for LAKLAK LAKLAK ([email protected]) Transaction: 11322

Figure 17.1: An Ice Cream as a Service API.

17.3.5 Grouping API URLs

Example 17.9: Project-Wide API Design

api/flavors/ # GET, POST

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 217

a URLConf called something like core/api_urls.py or core/apiv1_urls.py, and include that

Example 17.10: Combining Multiple App API Views Into One

from flavors.api import views as flavor_views

218 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

Prepared exclusively for LAKLAK LAKLAK ([email protected]) Transaction: 11322

17.3.6 Test Your API

17.3.7 Version Your API

17.3.8 Be Careful With Customized Authentication Schemes

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 219

ä Unless we are writing a non-cookie based scheme, we don’t disable CSRF.

TIP: Documentation Is Critical for Customized Authentication

17.4 When DRF Gets in the Way

17.4.1 Remote Procedure Calls vs REST APIs

In computer science terms, sundae.pour_syrup(syrup) could be classified as a Remote

220 Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues

Prepared exclusively for LAKLAK LAKLAK ([email protected]) Transaction: 11322

Example 17.11: Implementing pour_syrup() RPC with DRF

from rest_framework.response import Response

from ..models import Sundae, Syrup

def post(self, request, *args, **kwargs):

def get(self, request, *args, **kwargs)

And our API design would look like this now:

Please submit issues to github.com/feldroy/two- scoops- of- django- 3.x/issues 221

Example 17.12: Sundae and Syrup API Design

/sundae/ # GET, POST

17.4.2 Problems With Complex Data

Example 17.13: A Cone and Scoop API

/api/cones/ # GET, POST

│ ├── init.py