Web Development

Table of Contents

Amazon Web Services

EC2 (Virtual Servers)

General

Assigning an IP / Domain

Docker

Install

Install stable version with pacman -S docker

For Arch, the ‘loop’ kernel module had to be active.

The loop device is a kernel module that can be used to mount a file as a file system.

Source

Security concerns: Being added to the ‘Docker’ group is equivalent of getting root access. This is because the Docker daemon requires root privs to run. Users in the Docker group have access to this daemon. With access to said daemon, a user could share any directory on the host (including root /) with the container, giving someone accessing the container access to the entire host system.

The REST API endpoint (used by CLI to talk to daemon) now uses a UNIX socket instead of a TCP socket locally-bound to prevent this from happening.

Containers are created with a limited set of privs; binary root/non-root is turned into fine grained control system with ‘capabilities’. The ‘root’ in a container has fare less power than a true/host root. (no mounting, no raw sockets, no module loading)

Configuration

The default storage driver (graph driver), devicemapper, has shitty performance.

systemctl edit docker to create a ‘drop-in file’ for systemd. This will override/add new options on top of the original unit.

Usage

docker run <IMAGE> creates and runs the image specified in a container Depending on how the image wasa built, it could run a simple command and exit, or do much more.

    # what is this image based on?
    FROM docker/whalesay:latest
    # install commands
    RUN apt-get -y update && apt-get install -y fortunes
    # final command to run
    CMD /usr/games/fortune -a | cowsay

docker run <IMAGE> <COMMAND> runs that command and exits.

docker run -t -i ubuntu /bin/bash runs an interactive container. -t assigns a pseudo-tty/terminal inside the new container -i allows one to make an interactive connection by grabbing [STDIN] of container

Daemonize a container with docker run -d <IMAGE> <COMMAND>. -d runs container as a daemon. Command returns a container ID.

docker ps shows currently running processes; docker ps -a shows all Docker processes including ones that have exited

docker logs <CONTAINER> to show what is being sent to [STDOUT] in container Can use the -f flag to simulate tail -f to watch [STDOUT]

docker [start|stop] <CONTAINER> to start/stop container(s)

docker run flags:

Tips and Tricks

Create a non-root user

By default, Docker runs everything in a container as ‘root’ user with ID 0. If a user is able to break out of the container as root, they will enter the host as root too. Avoid this by running everything as a non-root user.

     RUN groupadd -r nodejs && useradd -m -r -g nodejs nodejs

     USER nodejs

Enable user namespace mapping

Avoid the previous issue of root jailbreak by running the Docker daemon with the following command:

dockerd --userns-remap=default

Leverage caching

Every RUN command in a Dockerfile creates a new ‘layer’ to capture files that were created/deleted/mutated between the beginning & end of said RUN command running.

Take advantage of this in regards to caching directories like node_modules. Simply copy the package.json file over to the container & run npm install before copying the app’s source files into the container.

     COPY package.json .
     RUN npm install --production
     COPY . .

Attaching to running container with new terminal session

If you want to connect to a container, using docker attach will just latch on to whatever processes are currently running. To do stuff in another terminal session without interruptiing what’s happening on the container:

docker exec -it <CONTAINER_NAME> /bin/bash

Creating Images

Two ways:

Networking Containers

Docker comes with two default network drivers: bridge and overlay. bridge: Limited to single host running Docker overlay: For multiple hosts, advanced usage By default, bridge is used as the default network for containers.

Disconnect container from network with docker network disconnect bridge <CONTAINER_NAME>

Create new bridge network with docker network create -d bridge <NETWORK_NAME> -d flags tells Docker to use the bridge driver. It’s the default value as well so it’s not needed.

Containers on different networks cannot see/talk to each other at all: not even ping.

Manage data in containers

Data volumes are designed to persist data independently of container. They are never deleted when removing containers.

Add data volumne to container with -v flag in docker [create|run] commands. Can be used multiple times to mount multiple volumes. Can also use the VOLUME instruction in a Dockerfile.

docker inspect <CONTAINER_NAME> will show mounted data volumes.

You can mount directories on host to container with -v <SRC_DIR>:<TARGET_VOL> eg: =docker run -d -P --name web -v /src/webapp:/webapp training/webapp python app.py This will mount /src/webapp on host to /webapp in container. must be an absolute path

If a path already exists on the container, the mounted path will overlay it but not remove it. It will become available when unmounting.

Add :ro to the end of the to mount in read-only mode instead of read-write

For mounting shared storage, install flocker and use the --volume-driver=flocker flag on docker run.

Best practice is to store persistent data in a data volume container that can be added to other containers at will.

First, create a new data volume container with docker create -v /dbdata --name dbstore training/postgres /bin/true This creates a new container named dbstore , based off of the training/postgres image with the volume /dbdata

Then use --volumes-from to mount the /dbdata volume in another container: docker run -d --volume-from dbstore --name db1 training/postgres

Removing containers that contain mount volumes will not delete said volumes. They must be explicitly deleted.

Django

Database (PostgreSQL)

Creating the database:

CREATE USER db_user;

Creating the user:

CREATE DATABASE db_name OWNER db_user;

For Postgres, we need to install the following dependencies:

pip install psycopg2

Set the DATABASES.ENGINE value in `settings.py` to ‘django.db.backends.postgresql’. Add USER, PASSWORD, AND HOST fields to DATABASE as well.

Virtual Environments

Best practice is to keep everything in a folder such as `~/.virtualenvs`. Honestly, just use `virtualenvwrapper`. It’s easier. Installable via pip (or AUR).

Views & URLs

Views go into `/views.py`.

In its basic form, a view takes a request:HttpRequest and returns an HttpResponse.

`django.shortcuts.render` can be returned also as a shortcut to render a template. It takes the following params: request:HttpRequest, templatePath:string, context:Dictionary

`django.shortcuts.get_object_or_404` can be used when getting data from db to 404 if no object is found. It takes a Django model as the first argument, and kwargs that get passed to get() of model There is also `get_list_or_404()`

When setting an internal URL, use the following template markup to reference a view by name kwarg:

<li><a href="{% url 'detail' question.id %}">{{ question.question_text }}</a></li>

If the app in question has namespaced urls by setting the `app_name` variable in `urls.py`, then you have to prefix the view name with `appname:` eg:

<li><a href="{% url 'polls:detail' question.id %}">{{ question.question_text }}</a></li>

POST data is accessible on `request.POST[‘name’]`.

Use `django.urls.reverse` in views when wanting to construct a URL using app:view to avoid hardcoding URLs.

Generic views take a lot of boilerplate / repition out of writing views like lists or objects, or displaying one object. `django.views.generic` has ListViews and DetailViews that can be used. Detail view needs the `model` and `template_name` as a string, and expects the key value captured to be called `pk`.

Once a view has been created, it needs to be hooked up to a URL. This happens in two steps. First, the package’s `urls.py`, found in the original folder created with the `startproject` command, needs to aware of our app. This is the main router for the project. In our package’s `urls.py`, we would do the following:

# other impoprts
   from django.conf.urls import include, url

   urlpatterns = [
       url(r'^polls/', include('polls.urls')),
       url(r'^admin/', admin.site.urls),
   ]

The ‘include’ method needs to be imported from `django.conf.urls`. We use it to reference our `.urls` file. If we want `blah/` to be the rool URL for the `blah` app, this is how we create that hookup. Everything after the trailing slash in the regex is chopped off and sent to the router we referenced.

Now we create a `urls.py` in the app directory. From this point on, when adding a new route to said app, we should only need to edit /urls.py.

from django.conf.urls import url

   from . import views

   urlpatterns = [
       url(r'^$', views.index, name='index'),
   ]

Import the package’s views module, and then pass the view & view’s name. The name allows use to refer to this view from other places within Django.

To capture a router parameter, you need to use a RegEx capture group in the urlpattern for said route. You also need to use a special type of pattern to capture the parameter to a variable in the view:

url(r'^(?P<variable_name>[0-9]+)/$', views.test, name='test')

The above will capture a number of at least one digit to the variable ‘variable_name’.

Models

Your Django project needs to know about your app. Add it to `INSTALLED_APPS` in `settings.py`:

INSTALLED_APPS = [
    'polls.apps.PollsConfig',
    '...'
]

   This refers to the module PollsConfig, in `apps.py` in the polls app.

Models are single source of truth about the data. They are placed in `/models.py>.

Models are classes that subclass `django.db.models.Model`. Adding a `__str__` method is helpful for debugging. Just return the name or human-readable ID.

One must run `python manage.py makemigrations ` to commit model changes to code. Then run `python manage.py migrate`.

The model API can be accessed after migrations at .objects.. eg: Question.objects.all()|count()|.create()|.get(), q.save(),

To add the model to the admin site:

admin.site.register(ModelClass)

Templating

To use static files:

{% load static %}
<link rel="stylesheet" type="text/css" href="{% static 'polls/style.css' %}" />

Testing

In /tests.py is where the tests lie.

Write a class for a certain group of tests & subclass django.test.TestCase. Methods in class can use self.assertIs & other methods for asserts.

To run tests:

python manage.py test <app>

For testing views, subclass `django.test.TestCase` to have access to `self.client.get/post/etc`.

Multi-tenancy

django-tenant-schemas app is the best option. As of Sept 3rd, seems to still have some issues with the latest version of Django.

Message Queues

Use Celery

Misc

Use Celery for async tasks & cron jobs

Django-Compressor for static compresion

Use Gunicorn instead of Apache???

Redis for session store

Sentry for monitoring

Replace default admin theme with something like grappelli or suit

/myproject /apps /configs /settings /deploy /static /templates urls.py fabfile.py

Two Scoops of Django

1: Coding Style

  • Use a project like flake8 to enforce PEP8 (the official style guide for Python)
  • Our Django project’s imports should be grouped in this order:
    1. Standard library imports
    2. Imports from core Django
    3. Related third-party imports
    4. Local application or library specific imports
  • Use explicit relative imports:

    # don't do this
      from cones.modelsl import WaffleCone
      from core.views import FoodMixin
      # do this
      from .modelsl import WaffleCone
      from core.views import FoodMixin

    Without explicit relative imports, changing the app name would require going through each file and changing the imports.

  • Avoid import * as importing two different modules may lead to conflicting names for what is imported (e.g.: django.forms and django.models both haave a class called CharField. The latter import will most likely be the version of CharField we end up with when using import *.)

  • Use underscores instead of hyphens in url pattern names (the names, not the patterns themselves)

2: The Optimal Django Envrionment Setup

  • Use Postgres in both development & production
  • Use pip and virtualenv to keep packages isolated to a single project
    • virtualenvwrapper is a tool that makes working with virtualenv less of “a pain to use”.
  • Install Django & dependencies via pip, use a requirements.txt file
  • Use a VCS (duh)
  • Docker can be useful, especially if not developing on a Linux host to begin with, or to ensure all team memberes are developing against the same environment

3: How to Lay Out Django Projects

  • Two Scoops recommends against the default project layout, saying it doesn’t scale well past the initial tutorial project. They recommend having the Django project root sit beside a configuration root folder, both inside the root of the project repository
    • Repository root: Contains READMEs, manage.py, .gitignore, requirements.txt and other high level files need for running the project & deploying
      • Sometimes combined with the Django project root
    • Django project root: Non-configuration Python code.
    • Configuration root: Settings modules and base url.py. Should have an __init__.py file, making it a Python package
  • The authors’ have a cookiecutter project that does all the boilerplate setup

4: Fundamentals of Django App Design

  • Django project: A web application powered by Django
  • Django apps: Small libraries designed to represent a single aspect of a project. Django projects are made up of many Django apps.
  • INSTALLED_APPS: A list of Django apps used by a given project
  • Third-party Django packages: Pluggable, reusable Django apps packaged with Python packaging tools
  • Each Django app should be tightly focused on its task
  • e.g.: The Two Scoops Ice Cream shop app may contain the following apps:
    • A flavors app to track ice cream flavors & list them
    • A blog app for official Two Scoops blog
    • A events app to display listing of shop events
    • A shop app that allows store to sell pints by mail order
    • A tickets app which handles ticket sales for premium events
  • Keep app names to single word names that is obvious to what it does
  • It’s better to have many small apps rather than a few giant ones
  • Aside from the usual modules found in apps (forms.py, urls.py, views.py), there are a few useful ones that aren’t so common:
    • api/: Package for isolating mdoules needed for creating APIs
    • behaviors.py: Model mixins can go here
    • constants.py: App-level settings can go here
    • decorators.py: Decorators
    • factories.py: Test data factories
    • helpers.py: Code extracted from views and models, can also be utils.py

5: Settings and Requirements Files

  • All settings should be version controlled
  • Inherit from a base settings file instead of repeating oneself
  • Keep secrets out of setting files
  • Keep a settings/ directory containing all settings files:
    • base.py: Settings common to all instances
    • local.py: Used for local development
    • staging.py: For clients to look at before production
    • test.py: Settings for tests
    • production.py: Production-level settings
    • ~ci.py: Contains settings related to CI (optional)
  • To run manage.py against a specific settings file:

    python manage.py [command] --settings=project_pkg.settings.local
  • Another option for using non-standard settings files is to use the DJANGO_SETTINGS_MODULE and PYTHONPATH environmental variables together

  • Keep sensitive secrets in environmental variables rather than direclty in setting files

    • One place to put them can be in virtualenv’s bin/postactivate script
    • Access these environmental variables on the Python side like so:

      import os
      os.environ['ENV_KEY_NAME']
  • If an environmental variable isn’t defined and one tries to access it via os.environ, an unhelpful KeyError will be thrown. Put the following snippet in settings/base.py and use it to access environmental variables to get a saner trace log:

    from django.core.exceptions import ImproperlyConfigured
    
        def get_env_variable(var_name):
            """Get the environment variable or return exception."""
            try:
                return os.environ[var_name]
            except KeyError:
                error_msg = 'Set the {} environment variable'.format(var_name)
                raise ImproperlyConfigured(error_msg)
    • We’ll be using django-environ which does this & loads environment variables from .env files
    • Don’t import Django components into settings modules
    • They recommend to use django-admin over manage.py when working with multiple settings files
    • Installing requirements from requirement files: pip install -r requirements/local.txt

6: Model Best Practices

  • django-model-utils and django-extensions are recommended packages to make dealing with models easier.
  • No more than 5 models per app recommended
  • Django offers three ways to do model inheritence:
    • Abstract base classes: Tables only creates for derived models. Has common fields in one place to prevent repeating
    • Multi-table inheritence: Tables creates for both parent and child. Implied OneToOneField between them. TwoScoops recommends against this
    • Proxy model: Table only created for original. Alias a model with different Python behavior
  • If overlap is minimal, avoid inheritence completely
  • Use abstract base model if enough overlap occurs that maintenance of separate repeated fields causes issues
  • Timestamping is common in Django, so creating a TimeStampedModel abstract model is useful.
  • When a new app or model is created, new django.db.migrations need to be created. Use python manage.py makemigrations
  • TwoScoops suggest that all models should start off normalized
  • Only denormalize if needed. Explore caching before turning to denormalization
  • TwoScoops has quite a bit of recommendations when it comes to when to use null=True and blank=True:
    • CharField, TextField, other text-like fields
      • null=True: Okay if both blank=True and unique=True to avoid unique constraints when saving empty values
      • blank=True: Okay if the widget should accept empty values. if null=True, values are stored in db as NULL, if not an empty string is stored
    • FileField, ImageField
      • null=True: Don’t
      • blank=True: Okay if the widget should accept empty values. if null=True, values are stored in db as NULL, if not an empty string is stored
    • BooleanField: Don’t
    • Number-related fields
      • null=True: Ok if colulmn can be null
      • blank=True: Okay if the widget should accept empty values. if null=True, values are stored in db as NULL, if not an empty string is stored
    • All other types that aren’t listed here follow ‘number-related fields’ rules
    • Avoid models.fields.GenericForeignKey - NoSQLish, not useful in most cases
    • Add model choices to the model as constants - this makes it easier to refer to those choices for operations such as filtering
    • _meta API is useful for a small subset of problems, such as:
      • Getting a list of a model’s fields
      • Get the class of a particular field for model (reflection)
    • Using Django’s ORM to query a model means we’re using an interface called a model manager to interact with the DB
      • We can create custom model managers by creating a class inherited from django.db.models.Manager and then assign an instance to that class to a models’s objects field
      • If using multiple managers on a model, make sure that objects = models.Manager() is set above any other manager that has a new name
    • Fat models is when we put all data-related code in model methods, class methods, properties and manager methods. Doing this means that all views or tasks can take advantage of these methods easily
      • Keep as much logic out of views and templates as possible
    • In order to make sure models don’t become too big, isolate code that will probably be reused. They are isolated into model behaviors or stateless helper functions
    • Model behaviors aka mixins is the idea of using composition to define what a model should consist of. Inherit from whatever abstract classes that define the mixins to get the fields needed & to avoid repetition
    • Stateless helper functions are essentially utility functions. They are stateless because all arguments need to be passed

7: Queries and the Database Layer

  • Use get_object_or_404() to get single objects - only to be used in views!
  • For all other queries, make sure that they don’t throw exceptions
    • get() will throw Model.DoesNotExist or ObjectDoesNotExist if it fails to return anything
    • If you want to throw if >1 object is returned, check Model.MultipleObjectsReturned
  • Django ORM doesn’t execute SQL until the data is actually needed (e.g.: iterating through a collection)
    • Lazy evaluation makes writing each query on a different line easy, makes it easier to read code
    • Chaining queries can be easy to read… this section is confusing?
  • Try to do advanced manipulations in the DB layer rather than in the web app - DBs are faster than most languages (especially interpreted ones)
    • Make use of query expressions, to continue doing heavy lifting in the DB
    • Use DB functions such as UPPER(), LOWER(), COALESCE(), CONCAT(), LENGTH() and SUBSTR()
  • Only drop down to raw SQL if it’s absolutely necessary!
  • Only add indexes to a field if:
    • That index will be used frequently (10-25% of all queries)
    • We have real data that shows that our bottlenecks are caused by lack of indices
    • Postgres: pg_stat_activity tells us what indices are being used
  • By default, Django ORM does not use transactions - each query is autocommited after executed
  • Set ATOMIC_REQUESTS in a DATABASE definition to True to enable transactions on all requests
    • This can cause performance slowdown and whether this is the best solution depends on individual DB design & usage
    • According to Django contributor Aymeric Augustin, this should be fine on most sites
  • The @transaction.non_atomic_requests decorator can be applied to a view to have its queries autocommit
  • with transaction.atomic(): blocks can be created to have specific DB operations wrapped in a transaction
  • Generally, DB operations that modify data should be wrapped in transactions, while read-only operations should not
  • Views returning http.StreamingHttpResponse can’t handle transaction errors if generating the response creates additional writes

8: Function and Class-Based Views

  • Function-based views: Normal function views that we all know
  • Class-based views: Generic, inherit from class
  • TwoScoops recommends class-based views unless we’re implmeneting a custom error view or a complicated view that would be too difficult to fit into a class-based view (see diagram on pg. 104)
  • Keep logic out of URLConf (duh)
    • This includes lines like DetailView.as_view(...), which defines a relationship between a view, model and template
    • Define the class views in views.py so they have their models & templates assigned, so we can use ViewName.as_view() from anywhere (namely, one or more urls.py), have the classes inherit from other classes
  • URL namespaces (adding namespace key to call to url()) are useful as they make shorter, obvious and more DRY url names, and decreases chances of a name conflict with a 3rd party library
  • Remember to try to keep business logic out of views and inside model methods, manager methods or utility functions
    • Hard to do at the start of a project so if you find yourself duplicating business logic in views, move it to somewhere more general
  • Simple view examples:

    • Function based view:

      from django.http import HttpResponse
      
          def simple_view(request):
              return HttpResponse('HI')
    • Class based views:

      from django.views.generic import ViewName
      
          class SimpleView(View):
            def get(self, request, *args, **kwargs):
              return HttpResponse('HI')
  • Don’t use locals() as view contexts - it makes it unclear as to what is being passed into the view. Explicitly pass context into views.

9: Best Practices for Function-Based Views

  • TwoScoop’s FBV guidelines:
    • Less view code is better
    • Stay DRY in views
    • Views should handle presentation logic; keep business logic in models if possible
    • Use FBVs to write custom error handlers
    • Avoid complex nested if blocks
  • Instead of passing a bunch of arguments to view utility methods, just pass the request object
  • Utility methods that do things like check permissions can simply return the request object as a pass condition, and raise an exception if failed
  • Decorators are nice but don’t go overboard with them; they can obfuscate what is actually happening and make it harder to reason about what the code is doing
  • Just like we can pass the HttpRequest around, we can also pass the HttpResponse around to other utility functions or decorators

10: Best Practices for Class-Based Views

  • Django views are just “a callable that accepts a request object and returns a response” (pg. 125)
    • Class-based views, being classes, have a as_view() method that returns the callable
  • django.braces library provides mixins for generic class-based views that Django lacks (although some of this is noew in core Django)
  • TwoScoop’s CBV guidelines:
    • Less view code is better
    • Stay DRY in views
    • Views should handle presentation logic; keep business logic in models if possible
    • Keep views simple
    • Keep mixins even simpler
    • http://ccbv.co.uk/ is a useful site for CBV documentation - more so than the official docs according to TwoScoops
    • When composing mixing in views, remember these rules of inheritence:
      • The base class provided by Django always go to the right
      • Mixins go to the left of base view
      • Mixins should inherit from Python’s built-in object type
    • When to use each type of generic CBV:
      • View: Base view, can be used for anything
      • RedirectView: Redirects user to another url
      • TemplateView Display a Django HTML template (good for static pages?)
      • ListView: List objects
      • DetailView: Display an object
      • FormView: Submit a form
      • CreateView: Create an object
      • UpdateView: Update an object
      • DeleteView: Delete an object
      • Generic date views: For display of objects over a range
    • 3 schools of thought for GCBV/CBV usage:
      • “Use all the generic views!” (TwoScoops)
      • “Just use django.views.generic.View, nice when “use all views” break down
      • “Avoid them unless views are being subclasses”
    • Use the django.contrib.auth.mixins.LoginRequiredMixin to only allow access to authenticated users
      • If you override dispatch() and use this method, make sure the first thing called in it is super(Class, self).dispatch(request, *args, **kwargs) or anything before it will be executed even if the user isn’t authenticated
    • With mixins, methods can be added to the view object itself - these methods & properties can be even accessed from within that view’s templates
    • Very useful Views + ModelForm example starting on pg. 135
      • Clever use of GCBV, flash messages and mixins
    • Very useful Views + Django form example starting on pg. 139
      • Creating a search page with a single ListView by modifying its get_queryset() method
      • Uses query parameters to return a different query set from the base one.. smart!
    • Using django.views.generic.View is similar to FBVs but instead of using nested if statements for different HTTP method handling, different methods are implemented on the class
    • Using View Class to create PDF on pg. 142

11: Common Patterns for Forms

  • Useful form packages
    • django-flopppyforms for rendering inputs in HTML5
    • One of:
      • django-crispy-forms for advanced form layout control in Bootstrap style
      • django-forms-bootstrap for simple Bootstrap-style forms
  • Pattern 1: Simple ModelForm with Default Validators
    • Creating a CBV with a model assigned will use default field validation rules of the assigned model
  • Pattern 2: Custom Form Field Validators in ModelForms
    • Custom validation on fields (e.g.: Ensure a field starts with a word)
    • Create validators.py module to put validator functions that return django.core.exceptions.ValidationError(msg) if not valid
    • Validators can be added to fields with the kwarg validators[validtor, ...]
  • Pattern 3: Overriding the Clean Stage of Validation
    • clean() and clean_<field_name>() are called after the validators are run
    • clean(): Validate two fields against each other
  • Pattern 4: Hacking Form Fields
    • A ModelForm’s __init__() method can be used to change, add or remove fields in a form. (e.g.: Making some fields required)
    • Can inherit one ModelForm from another to keep things DRY
  • Pattern 5: Reusable Search Mixin View
    • Similar pattern to the Search mixin from last chapter, but now attaching that mixin to the CBV with different models attached

12: Form Fundamentals

  • Validate all incoming data with Django forms
  • Always use CSRF protection when forms use POST
    • When posting data via AJAX, a client-side header named X-CSRFToken needs to be set
  • Add something to the Django form instance by having the first line of a ModelForm’s __init()~ method to be something like self.user = kwargs.pop('user') - do this before calling super()!
    • Then in the view, override get_form_kwargs()
    • django-braces will already auto-inject request.user into forms
  • What happens when form.is_valid() is called?
    • If form has bound data, full_clean() is called, which iterates through each form field which validates itself
      • Data coming into field is coierced into Python via to_python() or raised ValidationError
      • Data is validated against field-specific rules (incl. custom validators)
      • clean_<FIELD>() methods are called
    • full_clean() then calls form.clean()
    • If a ModelForm, form._post_clean() does:
      • Sets ModelForm data to Model instance
      • Calls model’s clean() method
    • ModelForm’s first save form data to the form instance, then the model instance
    • Form.clean() can be streamlined with Form.add_error()
    • Overriding HTML of built-in widgets on pg. 174
    • Creating new widgets on pg. 175

13: Templates: Best Practices

  • Supports Jinja2 out of the box since 1.8
  • Keep majority of templates in main /templates directory
    • This includes base.html along with subdirectories for each app
    • IMO, this makes more sense then the <app_name>/templates/<app_name> method recommended by Django
    • The exception to this is apps that are installed as pluggable packages
  • There are different architecture patterns when it comes to templates:
    • Two tier: All templates inherit from single root base.html
    • Three tier: Each app has a base_<app_name>.html which all inherit from a root base.html. Templates within apps all inherit from base_<app_name>.html. All other high level templates inherit from the root base.html
      • Best for sites where each section has a distinct layout
  • Flat hierarchies are easier to deal with than nested ones
    • This does not mean avoid using reusable blocks
  • Do as little processing in templates as possible
    • Looping over large querysets is probably a bad idea
    • Are the objects being retrieved large? If so, is every field being used?
    • Stick to the approach of templates being used to only display data that has been processed
    • Don’t filter with conditionals in templates - do this as the DB layer. Templates layer is not built for that kind of computation while the DB layer was
    • Avoid implied queries in templates - remember the Django ORM is lazy loaded so make sure the view context has all the data needed for that render fetched beforehand (or the use ORM’s select_related() method
      • Be careful about putting too much query logic in model methods & using those from templates
    • Some template tags use lots of CPU, like ones that manipulate images - be careful with these
    • Don’t do API calls in templates - put it in JS that executes after the template loads, or in a message queue/another thread in Python code
  • Inheritence in templates works by overwriting blocks defined in parent templates
    • Call {{ block.super }}~ in templates to render the parent template’s block contents before the child
  • Don’t couple styles tightly to Python code (duh!)
  • Underscores over dashes in names, etc
  • Include the name of the block in the endblock for clearer reading
  • Use URL names instead of hard coded paths
  • Force more verbose error messages by seting OPTIONS.string_if_invalid to "INVALID EXPRESSION: %s" inside of TEMPLATE options
  • It’s standard too create at least a 404.html and 500.html for error handling pages
    • These pages should have all CSS/JS inline to have the chances of a render failure decrease

14: Template Tags and Filters

  • A filter is just a function that takes up to two arguments
  • Filters themselves should probably call a utility method defined elsewhere to increase code reuse
  • Don’t overuse template tags - especially custom ones
  • Template tags are hard to debug
  • Template tags can have a big performance cost
  • Anything that causes a read/write of data would be better served in a model or object method
  • Use template tags when they are only resonsible for rendering HTML
  • Name your tags <app_name>_tags.py
  • Load template tags with the load tag keyword like so: {% load <TAG_NAME> %}
  • Don’t implicitly load template tags with OPTIONS.builtin in settings.py

15: Django Templates and Jinja2

  • Jinja2 is an optional template engine built-in to Django
  • Jinja2 has some differences, like method calls having () parens at the end, as well as some keywprds have different names
  • We don’t have to chose one or the other - we can use both.
  • Advantages of Django Template Library (DTL):
    • Works by default, documentation clearly outlined in Django docs
    • More mature than Jinja2
    • Most third-party packages use DTL
    • Converting a large code base from DTL to Jinja2 sucks
  • Advantages of Jinja2
    • Independent of Django
    • Closer to Python syntax, more explicit
    • Less arbitrary restrictions (e.g.: unlimited arguments to a filter)
    • Faster than DTL
  • Use DTL 99% of the time, use Jinja2 is performance is required and benchmarks show an improvement. However, stick one with ‘template type’ as a primary templating language
  • Jinja2 does CSRF differently than DTL. An input element with the value being set to csrf_token needs to explicitly be set
  • DTL template tags don’t work in Jinja2
  • pg. 212 explains how to use Django template filters in Jinja2
  • Jinja2’s Envvironment object should be considered static, as it initializes when the first template is loaded and doesn’t change much after

16: Building REST APIs With Django REST Framework

  • REST = Representational State Transfer
  • Best package for building RESTful APIs with Django is DRF: Django REST Framework
  • Leverages the best parts of Django (CBVs, ORM, out of the box authentication and authorization)
  • Has a huge community
  • Most of the chapter seems to be REST API fundamentals and basics
  • Useful HTTP status code chart on pg. 219
  • By default, setting ‘admin-only’ as the default permissions is a nice security default - then enable access to views on a per-view basis
  • Use UUID instead of sequential integers for a model’s primary key (?) for models with public lookups
  • Useful docs resource for DRF
  • Define API in <app_name>/api
    • Modules: authentication, parsers, permissions, renders, serializers, validators, views, viewsets
  • Sometimes it makes sense to build the API as a Django app rather than a Python package
  • If views get too big, break the views up into its own package
  • Just like we keep business logic out of normal views, keep business logic out of API views as well
  • Make sure to implement rate limiting on your API - it can be a part of the business plan
  • Using nginx or apache for rate limiting means better performance but removes control of this feature from Python

17: Consuming REST APIs

  • Talks about front-end tools/frameworks like React, Vue, jQ
  • Lots of high-level talk about front-end stuff.
  • Dealing with latency from HTTP requests:
    • Mask with animations
    • Fake a successful transaction (send it in the background, report that it finished whe it’s still in progress)
    • Servers all over the world (expensive)
  • DRF has a client side library that makes interacting with DRF easy: http://www.django-rest-framework.org/topics/api-clients/#javascript-client-library
  • If our API accepts cookies, never disable CSRF protection!
  • if CSRF_COOKIE_HTTPONLY is True, we need to embed the token in a hidden input tag which we can then extract the token text out of

18: Tradeoffs of Replacing Core Components

  • Replacing core Django components is definitely possible but is probably not worth it
  • I’ll most likely never have to deal with any of these issues

19: Working With the Django Admin

  • Easier to create custom management dashboards than to make admin dashboard fit client needs
  • Overwrite a model’s __str__() method to get a better string representation of itself in the admin panel
  • We can also use the list_display of a ModelAdmin, which takes a tuple of field names to show
  • We can add callables to a ModelAdmin functions that will either modify the list or display something. See pg. 261 for implementation details
  • Multiple users logged into the panel at the same time can have race conditions (e.g.: User A edits model but doesn’t save, User B edits models and saves, User A comes back and saves their model - User B’s changes are now overwritten)
  • django.contrib.admindocs is a useful tool to generate docs based off docstrings in the project
  • Django Admin has some custom skins listed on pg. 265
  • Changing the default admin panel URL is an easy way to secure it
  • django-admin-honeyput puts a fake login at admin/ and “logs info about anyone who attempts to log in”
  • Only allow access via HTTPS
  • Secure the admin docs as well by changing its default URL

20: Dealing With the User Model

  • Advised way to get user class is with django.contrib.auth.get_user_model
  • The preferred way to attach a foreign key is as follows:

    from django.conf import settings
    from django.db import models
    
      class IceCreamStore(models.Model):
          owner = models.OneToOneField(settings.AUTH_USER_MODEL)
          title = models.CharField(max_length=255)
  • Creating our own user model can be done in a few ways:

    • Subclass AbstractUser: If you like like the default User fields but need a few extra, use this. Set AUTH_USER_MODEL to the new model package.
    • Subclass AbstractBaseUser: If you aren’t happy with the default User fields. Only comes with 3 fields, password, last_login and is_active.
    • Linking Back From a Related Model: Legacy way of doing things. Use if creating a third party projects, or if your internal projects has different kinds of users or other weird edge cases.

21: Django’s Secret Sauce: Third-Party Packages

  • Make use of all the awesome third-party packages
  • PyPI (Python Package Index) is a repo of Python packages. Where pip retrieves packages from
  • djangopackages.org is “a directory of reusable apps, sites, tools and more for your Django projects” (pg. 279)
  • When you find a package you want to use:
    • Make sure the docs don’t suck
    • Add package & version number to requirements file
    • Run pip install to install into our virtual envrionment
    • Follow the package install instructions

22: Testing Stinks and Is a Waste of Money!

  • coverage.py for code coverage
  • Delete the default tests.py created with new apps, create a tests package instead containing test_forms.py, test_models.py, test_views.py. Any other files that need testing should have a corresponding test file created for it
    • Tests must be prefixed with test_ for Django’s test runner to see them
  • When creating unit tests for views, use the setUp() method in TestCase classes to setup the minimal environment needed to test a view (e.g.: create records that will be retrieved)
  • Use django.test.RequestFactory when possible
    • “[It] provides a way to generate a request instance that can be used as the first argument to any view” (pg 302)
    • Would be added as a property to the TestClass in the setUp() method
    • Mocking the request object, essentially
  • Tests don’t always need to be DRY (Personally, I don’t think TwoScoops offers any real justification for this aside from “it’s bad”)
  • Fixtures are hard to maintain over a project’s lifespan so don’t use them, so use other tools to generate test data:
    • factory boy
    • faker
    • model mommy
    • mock
  • Test everything (Hmmm)
    • Views: Viewing data, changing data
    • Models: CRUD, model methods, manager methods
    • Forms: Methods, custom fields, clean()
    • Validators: Fuzzing
    • Signals
    • Filters
    • Template tags
    • Everything else
    • Failure states for all the above
  • Unit tests should mock anything external to the current unit
  • Integration tests are useful too (not a lot of detail about them in the Django context though)
  • Tests are especially useful when upgrading Django or third party packages
  • How to gamify code coverage (pg 312)

23: Documentation: Be Obsessed

  • RST (ReStructuredText) is the standard markup language in the community
  • Similar to normal Markdown in some ways although links are different, as are numbered lists #) and code blocks
  • Use Sphinx to generate documentaion
  • Nice structure to have for docs:
    • README.rst: The README
    • docs/
      • deployment.rst: How to deploy
      • installation: How to install a dev. environment
      • architecture: Design decisions
  • Can always use Pandoc to convert between Markdown and RST

24: Finding and Reducing Bottlenecks

  • Premature optimization is bad. None of this is necessary for small projects
  • The django-debug-bar is useful to see what queries are being run
  • Other tools for profiling:
    • django-cache-panel
    • django-extensions can run runserver with a profiler attached
    • silk is a live profiling Django app
  • If you have a part of the site with too many queries - reduce them!
    • Try to use select_related() to combine queries - but don’t let them get too big
    • If the same query is being generated more than once per template, move the query to the Python view and add to the context
    • Implement caching
  • Put SQL indexes on fields that are filter/sorted by most frequently
  • Profile at the SQL level
  • Switch ATOMIC_REQUESTS to False
  • Things that shouldn’t go into a DB:
    • Logs
    • Ephemeral data
    • Binary data
  • Recommended articles on Postgres tuning (pg 329)
  • Cache queries with Memached or Redis - it can help a lot
  • There are some third party packages for caching, but they aren’t a silver bullet
    • django-cache-machine
    • johnny-cache (hah)
    • django-cachalot
  • Compress/minify HTML/CSS/JS
  • Use a CDN

25: Asynchronous Task Queues

  • Async tasks are are executed at a different time from when they are created
  • Broker: Storage for the tasks themselves, usually something like RabbitMQ or Redis
  • Producer: The code that adds the task to the queue. This code usually lives in the application we are developiong.
  • Worker: Code that takes tasks from the broker and executes them. Usually more than one worker.
  • Serverless: Services where one writes some server side logic that is run in a stateless container and is ephemeral, and 3rd party managed. AWS Lambda is an example
  • Use task queues if results can take time to process
    • Sending bulk emails
    • Modifying files
    • Fetching large amounts of data from 3rd party APIS
    • Upserting a lot of records into a table
    • Performing time-sensitive calculations
    • Sending/receiving webhooks
  • Smaller sites may never need to use a task queues while larger ones may need one for every action
  • Various task queue options such as Celery (which I’ve used before), DjangoChannels (no retry mechanism), AWSLambda (require external API calls), Redis, django-background-tasks (bad for anything medium-to-high)
  • TwoScoops recommends AWS Lambda and moving to Celery if that’s too slow (not sure if I agree personally)
  • Use Django channels for websockets
  • Treat tasks like views - keep them thin, and keep models fat
    • Write in a way that would make porting a task from Channels to Celery easy
  • Only pass JSON-serializable values to task functions - it can be hard to debug complex objects being passed to a task
  • Idempotent: Run task multiple times but result isn’t applied past initial execution
    • Write pure functions (no side effects) over idempontent functions
  • Don’t put important data in a queue /(is the name of this section, but instead of telling us where to put it, the authors tell us how to avoid an edge case where a task fails. Wat?)/
  • Periodically clear out dead tasks

26: Security Best Practices

  • Make sure the server hosting the product is hardened against attacks - out of scope for this book
  • Django’s included security features
    • XSS protection
    • CSRF protection
    • SQL injections
    • Clickjacking
    • Support for TLS/HTTPS/HSTS
    • Secure passwords
    • Auto HTML escaping
    • Expat (?) parser
    • JSON/YAML/XML [de]serialization tools are hardened
  • Obvious things like turning off DEBUG in prod, keeping keys in secret (in .env files), use HTTPS everywhere
  • /(Bit of hyperbole on Let’s Encrypt. While I think it’s great, buying SSL certs before wasn’t some crazy arcane and painful process. Also, Let’s Encrypt doesn’t offer wildcart certs so it doesn’t cover every usecase)/
  • Better to enforce HTTPS as a web server level rather than application level
  • Use secure cookies
  • Use HSTS
  • Use allowed hosts validation
  • Avoid cookie-based sessions
  • Validate all incoming data with Django Forms
  • Disable autocomplete on payment fields
  • Server user-uploaded content from a completely separate domain (via a CDN)
    • Use python-magic to check uploaded files headers
  • Never use ModelForms.Meta.exclude - will include all fields not listed, including sensitive info
  • Don’t store credit card data
  • Use defuedxml to defend against XML bombing
  • Explore 2FA with django-two-factor-auth
  • Use django.middlware.security.SecurityMiddleware to enable settings such as forcing strong passwords
  • Never dipslay sequential PKs - they inform people of our volume, provides targets for XSS attacks
    • Look up objects by slug
    • Use UUIDs instead via django.db.models.UUIDField

27: Logging: What’s It For, Anyway?

  • In production, use every type of log level except for DEBUG
    • The log levels: CRITICAL, ERROR, WARNING, INFO, DEBUG
    • CRITICAL: Something catastrophic has occured
    • ERROR: Production errors. Used by Django when an exception is raised by not caught. If you want an email if something fails, use this
    • WARNING: Not as bad as ERROR but we still want to keep track of it
    • INFO: If we need to log details that will be important/useful to look at later
    • DEBUG: Equivalent of a print() statement in other languages
  • Settings the kwarg exc_info to true in logging methods will attach a stack trace
  • Use one logger per module
  • Log INFO and above to rotating local log files
  • logutils is a useful package that has neat logging features like colourized console loggging, ability to log to queues and more

28: Signals: Use Cases and Avoidance Techniques

  • Use Signals as a last resort
  • Signals are synchronous and blocking
  • Only use signals over functions “if the piece of code sending it has positively no way to determine what its receivers will be.” (pg 382)
  • Useful for:
    • Dispatching same signal from multiple apps and have it handled in the same way
    • Invalidate cache after model save
    • Receiver makes changes to more than one model
  • Instead of using pre_save and post_save signals, just override a model’s save() method
  • Use helper functions instead of signals

29: What About Those Random Utilities?

  • Put all the random little utility and helper functions that can be used elsewhere in a Django app called core which contains functions and objects for use across a project
  • For methods that are just useful for a single app, just make a utils.py or helpers.py module in that Django app
  • Useful packages in core Django:
    • django.contrib.humanize: Makes some numbers or strings easier to read (e.g.: adding commas in a large number)
    • django.utils.decorators.method_decorator(decorator): Turn a function decorator into a method decorator
    • django.utils.decorators.decorator_from_middleware(middleware): Isolate use of middleware on a per-view basis by turning it into a decorator
    • django.utils.encoding.force_text(value): Turn whatever into a string
    • django.utils.functional.cached_property: Cache the result of a method with a single self argument as a property
    • More listed, check this chapter for details
  • Lots of different built-in Python exceptions listed on pg 395
  • We can set a 403 error page to any view we want with the following in the root URLConf:

    handler403 = 'core.views.my_permission_denied_view'
  • Django comes with some nice [de]serialization tools in django.core.serializers

  • django.core.serializers.json.DjangoJSONEncoder is useful as it can properly parse date/time or decimal types

  • rest_framework.serializers should be used if DjangoJSONEncoder still isn’t enough

30: Deployment: Platforms as a Service

  • AWS! Heroku! PythonAnywhere!
  • Several critera to evaluate when deciding on a PaaS are listed
  • Maintain identical environments everywhere: Docker helps with this
  • Maintain a staging instance
  • Automate deployments: a Makefile can make this easy

31: Deploying Django Projects

  • A single server (e.g.: DigitalOcean) is the quickest/cheapest way to get Django up and running, but still TwoScoops recommends EBS or Heroku
  • At the most basic level, have a DB esrver & application server
  • Other potential servers: static files (use CDN instead), caching/async queues, CPU-intense tasks
  • Advanced setup detailed starting on pg. 417
  • Always deploy with WSGI:
    • uWSGI & nginx
    • Gunicorn behind nginx
    • Apache & mod_wsgi
  • Most popular automation/deployment related tools: Docker, Kubernetes, Ansible, SaltStack

32: Continuous Integration

  • It’s a chapter about CI!
  • Auto run versions of the app against tests
  • Write lots of tests to take advantage of CI
  • Keep the builds fast /(I know how important this can be… one of my last contracts has a web application that took around ~20minutes to finish its CI.)/
    • Maybe write less integration tests and more unit tests
  • Tox is a tool that will allow “us to test our projects against multiple Python and Django versions with a single command at the shell.” (pg 433)

33: The Art of Debugging

  • Use django-debug-toolbar
  • __init() takes exactly 1 argument (2 given)__ is a common CBV-related error
    • Check your URLConf, is there a missing as_view() method call on the CBV?
  • PDB is a REPL / interface with interacting with source code at the specified break points
  • If there are problems with form file uploads, ask yourself the following:
    • Does <form> include an enctype attribute?
    • Does the associated view handle request.FILES?
  • To make debugging production applications easier, make sure logs are piped to a central place or use a tool to make them human readable, look for errors, etc
  • Clever creation of a UserBasedExceptionMiddleware to let superadmins see 500 error pages as if settings.DEBUG = true on prod on pg 444
  • Feature flags allow certain features to be enabled for a subset of users on production to allow for real testing & feedback before releasing the future to the general user base of an application
    • django-gargoyle and django-waffle are useful tools for creating feature flags in Django

34: Where and How to Ask Django Questions

  • IRC (#django@freenode), StackOverflow
  • Obvious (to me) stuff

35: Closing Thoughts

  • /Dale’s thoughts on Two Scoops of Django 1.11: I found the first half of the book to be very useful. While there were a couple of points I would have liked more explanation on (why or why not to use/avoid a certain pattern rather than just saying “don’t do it!”). The second half of the book is probably useful for a large subset of people but I personally didn’t feel I got a lot out of the second half. Overall, I think it was worth it./

Elixir

Basics

  • All types are immutable
  • Types:
    • Integer
    • Float
    • Boolean (true / false)
    • Atom / Symbol (:dale)
      • Constants which have their name as their value
      • is_atom(var), returns true or false
    • String
      • String concatination: "hello" <> " world"
      • Interpolation: "hello #{:world}"
      • Strings can have line breaks in them
      • Print a string with IO.puts/1
      • Get string length with static method String.length(string)
    • List ([1, 2, 3])
      • Values in a list can be of any type & be mixed
      • Use length [] to get length size of list
      • Concat lists with ++/2 or --/2
      • Head is first element of list, tail is the remainde of the list. Retrieve with functions hd/1 and tl/1
    • Tuple ({1, 2, 3})
      • Can hold any value like lists
      • Store elements contiguously in memory. Tuples are fast!
      • Indexes start at 0
      • Get elements of tuple out with elem/2, check length with tuple_size/1
      • Create a new tuple with an added element iwth put_elem/3
  • Lists vs tuples: Lists are stored as linked lists. Accessing length of list is a linear op.
  • Division / returns floats, use div(1, 2) for integer division or rem(1, 2) for modulo
  • Functions are identified by their name & arity (# of arguments a function takes)
  • Annonymous functions are delimited by keywords fn and end

      add = fn a, b -> a + b end
      add.(1,2)

    Argument list is followed by -> followed by function body. Also notice the dot between variable and paren in functin call. This is necessary to ensure no ambiguity between a named & annon. function.

  • Annonymous functions are closured. Can access variable in scope via lexical scope

  • Functions named size operate in constant time, functions named length operate in linear time

  • Other more advanced data types: Port, Reference, PID Next chapter!

“You Don’t Know JS”

This does not represent a full site of notes for “You Don’t Know JavaScript”. I wrote notes for things I found new or interesting.

Things I skipped over/may need looking over in the future:

  • Proxy / Reflect API
  • Iterables

Scopes & Closures

Chapter 1 - What is Scope?

  • JS is compiled on-the-fly (JIT) by the runtime envrionment (browser/Node). Not a lot of time to optimize.
  • Following steps are taken to go from script to machine code (standard compiler steps):
    1. Tokenizing/Lexing: Turn script/string of characters into chunks called tokens
    2. Parsing: Take stream of tokens and turn it into a tree of nested elements which represents the grammatical structure of program called the Abstract Syntax Tree
    3. Code Generation: Take AST and turn it into machine code
  • JS engine is responsible for the task of compilation & execution
  • The compiler handles parsing and code generation
  • Scope is responsible for collecting & maintaining a list of declared identifiers
  • Handling of statement var a = 2:
    • Compiler breaks it down into tokens, which then gets parsed into an AST.
    • During code generation:
      • var a: Compiler asks Scope to see if a already exists for that scope collection. If so, compiler ignores this declaration and moves on. If not, Compiler asks Scope to create a new variable called a
      • Compiler produces code for Engine to execute which handles the a = 2 assignment. Engine will ask Scope if a is accessible in current scope. If so, Engine will that varaible. If not, Engine looks elsewhere. If Engine eventually finds a variable, it will assign 2 to it
    • Dale note: Is this execution context vs normal execution?
  • When Engine is asking Scope whether a variable has been declared, it can perform a LHS or RHS lookup. (Is the variable on left or right of assigment operation?)
    • RHS: Get value of variable. Anytime a variable is being used by value, we can say its a RHS lookup
    • LHS: Get reference of variable and set the value
  • Quiz: Identify the 3 LHS lookups and the 4 RHS lookups:

    function foo(a) {
      var b = a; // LHS (b), RHS (a)
      return a + b; // RHS (a), RHS (b)
    }
    
    var c = foo( 2 ); // LHS (c), RHS (foo), LHS (2 -> a)
    
  • If a variable cannot be found in the immediate scope, Engine will ask the next outer containing scope. It will do this until the outermost scope (global) has been reached.

  • Why do we even care whether a variable is a left-hand or right-hand lookup? They behave differently when variable has not been declared (not found in Scope)

    • If a RHS lookup is made and the variable does not exist, a ReferenceError is thrown by the Engine
    • If a LHS lookup fails and strict mode is not enabled, it will create the variable in the last scope it looked in, which would always be the global scope
    • If a LHS lookup fails and strict mode is enabled, a ReferenceError will be thrown
    • If a RHS lookup succeeds but one tries the use the value in an “illegal” way (ie: use a non-function as a function, reference a prooperty on null or undefined), the engine will throw a TypeError
    • ReferenceError is Scope resolution-failure, TypeError implies Scope resolution succceeded but an illegal action is being taken against the result

Chapter 2 - Lexical Scope

  • Two predominant models on how scope works
    • Most common is lexical scope
    • Other model is dynamic scope, not used as much
  • Lexing: Convert string to tokens
  • Lexical scope: Scope defined at lexing time. In other words, scope is defined on where variables and block scopes are at the time of authoring.
  • As stated in the previous chapter, the Engine will look for identifiers up the scope chain - from local up the chain to global
  • Scope lookup starts at the innermost scope and stops at the first match. This enables the concept of shadowing, where the inner identifier is used over outside ones)
  • Lexical scope lookup only applies to first-class identifiers (a instead of object.foo.a)
  • Lexical scope is only defined by where the function was declared, not where or how it was invoked
  • “Cheating” lexical scope (by changing it at run-time) is frowned upon as it leads to poor performance

    • eval takes a string and treats that string as if it were originally authored at that point. strict mode changes this so eval operates within its own lexical scope and doesn’t modify the ecnlosing scope
    • with (a deprecated feature) is used to make multiple property references against an object without repeating the object reference each time.

      var obj = {
        a: 1,
        b: 2,
        c: 3
      };
      
      with (obj) {
        a = 3;
        b = 4;
        c = 5;
      }
      

      with takes an object and treats that object as if it is a wholly separate lexical scope. Passing an object to with that attempts to set a non-existent property on that object will cause a global variable leak as the Engine travels up the Scope to look for that identifier

  • To recap, eval modifies the lexical scope while with creates a new lexical scope out of nowhere

  • The performance hit that comes from modifying lexical scope at runtime comes from the fact that Engine compilation performs optimizations. If the JS engine sees eval or with, it wont perform these optimizations as the Engine has no way to guarantee the state of Scope at that point

Chapter 3 - Function vs. Block Scope

  • JS uses function-based scope. Each function creates its own scope.
  • Attempting to access identifiers from outside of a scope results in ReferenceError since they won’t be available on global scope either
  • Identifiers belong to a function or global scope
  • Wrapping a function around code you wrote ‘hides’ code since that code is not available from outside that function.
    • One reason to do this is based on the software design principle of “Principle of Least Priviledge” in which only what is necessary is exposed, and everything else is hidden
    • Another benefit of “hiding” identifiers inside a scope is to avoid identifier collisions - especially of libraries in the global scope. Libraries will define one variable with a unique identifier in the global scope and keep everything related to it under that namespace
  • Hiding code in a function usually requires a named function (polluting scope) and then calling that function so it executes. We can get around this with annonymous self-executing functions:

    var a = 2;
    
    (function foo(){
      var a = 3;
      console.log( a );
    })();
    
    console.log( a );
    
  • Function declaration: first token of statement is function, function expression: anything but function is first token of statement

  • In the above example, the identifier foo is not available in the top level scope but within its own function.

  • Function expresions are often used as callback paramters, usually with no name which makes them annonymous

    • Function expressions can omit names but function declarations cannot
  • Annonymous functions do have drawbacks

    • No useful name to display in stack traces
    • No way for function to refer to itself without use of the deprecated arguments.callee
    • Function names can be self-documenting in telling the reader what that function actually does
  • Giving function expressions a name has no downside, so just do it!

  • Back to the example: The function is wrapped in paren; adding another pair at the end allows us to execute it

  • This pattern is known as IIFE: Immediately Invoked Function Expression

  • Whether the invoking parens () are inside or outside the function-wrapping parens is a completely stylistic choice

  • Another variation of IIFE allows for passing arguments:

    (function IIFE(global){
      var a = 3;
      console.log( a ); // 3
      console.log( global.a ); // 2
    })(window);
    
  • Block scope did not exist in JS until recently except for weird caes, like using with to create a scope from the passed object

  • try / catch: the variable passed to catch is actually block scoped to the catch block (ES3)

  • The let keyword used to declare variables and attaches that variable to whatever block (defined by {}) the let statement is in:

    var foo = true;
    
    if (foo) {
      let bar = foo * 2;
      bar = something( bar );
      console.log( bar );
    }
    
    console.log( bar ); // ReferenceError
    
  • Hoisting does not apply to variables declared with let.

  • Block scoping is also useful for closures and letting the engine/garbage collector know what identifiers it doesn’t need to keep around

  • const is another keyword introduced with ES6 to define block-scoped variables except the value is fixed at declaration

  • let does not necessarily replace var function scope. They both have their uses.

Chapter 4 - Hoisting

  • Remember that during the compiler’s compilation phase, identifiers are added to a scope collection but the value is not yet assigned until runtime. For example:

    console.log(a);
    var a = 2;
    

    The reason this is undefined rather than a ReferenceError is because the a identifier was added to the Scope at compilation. However, until that line is ran by the engine, the value of a is undefined. This is known as hoisting.

  • Hoisting occurs per-scope

  • Function declarations, being a declaration, are hoisted while function expressions are not

    • Attempting to call a hoisted function expression would throw a TypeError since the variable it belongs to is declared but the value is undefined, and one cannot use undefined as a function
  • Functions are hoisted first, and then variables

    • Weird side effect: Subsequent functions declarations with the same identifier override previous declarations

      foo();
      
      function foo() {
        console.log( 1 );
      }
      
      var foo = function() {
        console.log( 2 );
      };
      
      function foo() {
        console.log( 3 );
      }
      

Chapter 5 - Scope Closure

  • Closures happen as a result of writing code that relies on lexical scope
  • Formal definition of closure: when a function is able to remember and access its lexical scope even when that function is executing outside its lexical scope.

    function foo() {
      var a = 2;
    
      function bar() {
        console.log( a );
      }
    
      return bar;
    }
    
    var baz = foo();
    
    baz(); // 2 -- Whoa, closure was just observed, man.
    

    In the above example, we would say that bar() closes over the scope of foo() because bar() appears inside of foo(). bar() is then returned outside of its lexical scope & executed while still retaining access to its lexical scope. The garbage collector will never remove foo() or its contents because it is still in use by the returned bar().

  • Closure allows functions to continue their lexical scope it was defined in at author-time from no matter where that function is

  • let declarations in a loop containing a function that closes over that loop will have that variable declared for each iteration

    for (let i=1; i<=5; i++) {
      setTimeout( function timer(){
        console.log( i );
      }, i*1000 );
    }
    
  • Closures are often utilized in the module pattern:

    function CoolModule() {
      var something = "cool";
      var another = [1, 2, 3];
    
      function doSomething() {
        console.log( something );
      }
    
      function doAnother() {
        console.log( another.join( " ! " ) );
      }
    
      return {
        doSomething: doSomething,
        doAnother: doAnother
      };
    }
    
    var foo = CoolModule();
    
    foo.doSomething();
    foo.doAnother();
    

    A public API is returned which retains access to its lexical scope (private variables). The initial call to CoolModule() assigns values to the declarations inside

  • The above pattern can also be wrapped in an IIFE function expression if one needs a ‘singleton’ pattern of sorts

  • ES6 adds module support with export / import keywords. No inline module format; one module per file

this & Object Prototypes

Chapter 1: this or That?

  • Function.prototype.call takes a list of arguments while Function.prototype.apply takes one array of arguments. The first argument of both sets the value of this
  • this is a special identifier keyword automatically defined in the scope of every function that is bound at runtime
  • this does not refer to a function’s lexical scope
  • The value of this is contextual and is based on how the function is invoked, not about where it is declared. When a function is invokved, an activation record known as an execution context is created. It contains info such as where it was called from, how it was called & parameters

Chapter 2: this All Makes Sense Now!

  • Call-site is how & where the function is called; this determines how this is defined
  • Call-stack: stack of functions that have been called to get us to current moment in execution
  • Standalone function invocation means this inside said function will resolve to the global object. strict mode will make it undefined
  • When a function is invokved as a method on an object, this will resolve to that object
  • When a function is called using call or aply, the value of this is the first argument of each. bind creates a new function with this being hardcoded to be whatever the argument is to bind
  • Be careful making references/aliases to methods and then calling them like plain functions; value of this will be lost
  • You can hard bind functions to values of this with ES5’s Function.prototype.bind, which returns a new function with this bound to whatever you desire
  • Functions invoked with the new keyword makes that function a constructor call. A constructor call invokes the following:
    1. Brand new object is created
    2. Newly constrcuted object is linked to Prototype
    3. Newly constrcuted object is set as this for that function call
    4. Unless function already returns an object, the new call will return newly constructed object
  • Order of precedence for this binding rules:
    1. new binding (constructor call)
    2. Explictic binding (call / apply / bind)
    3. Implicit binding (method call)
    4. Default binding (nromal function invoke)
  • ES6 arrow functions \=> allows said function to adopt the this binding from the ecnlosing scope (lexical scope). This gets rid of the need for tricks like var self = this
  • getify does not recommend mixing lexical scope coding with this style coding inside of the same function

Chapter 3: Objects

  • JS object literal syntax: {key: "value", ...}
  • Primary types in JS: string, number, boolean, null, undefined, object
  • Everything in JS is not an object
  • Built-in objects: String, Number, Boolean, Object, Function, Array, Date, RegExp, Error
    • JS automatically coerces string literals to the String object type, number literals to Numbers, bool literals to Boolean
  • Engine will store “contents” of object in implementation-specific ways; what is stored on the object is the property names, which act as references to where the values are stored
  • Object property names are always strings; numbers will be coerced to string
  • ES6 adds computed property names with an expression surrounded by [] in the key position of an object literal
  • JS doesn’t really have ‘methods’ - invoking a function using the “method” format doesn’t mean that that function “belongs” to that object
  • Shallow copy objects with Object.assign or by stringifying and then parsing to JSON
  • ES5 added Object.defineProperty to explicity define properties with some metaddata, such as whether that property is read-only, or whether the propert configuration can be re-defined/deleted (configurable)
  • ES5 added methods to make an object immutable in a shallow way (changes to references are not immutable)

    • Object.preventExtensions prevents new properties from being added
    • Object.seal will use Object.preventExtensions in addition to making all properties configurable:false
    • Object.freeze will perform the same actions as Object.seal but also make all properties writable:false
    • When performing a property access on an object, a \[\[Get\]\] operation is performed on the object, which will first look at the object for the property and returns it if found; if not, it will go up the prototype chain and return undefined if not found
    • We can define new get / set functions in Object.defineProperty for individiual properties on an object, or as an object literal:

      var myObject = {
        // define a getter for `a`
        get a() {
          return 2;
        }
      };
      
      myObject.a = 3;
      
      console.log(myObject.a);
      
    • If you’re going to define a setter or getter, define the other as well; using only one can lead to undefined behaviour

    • To check if a property belongs to an object, use property in object if you don’t care about looking up the scope chain or hasOwnProperty if you only want to search the given object

    • Use for keyvar in object to iterate over an object’s properties. Use propertyIsEnumerable to check if a property can be iterated over

    • Object.keys returns all of an object’s keys in an array

    • Iterate over arrays with forEach, every and some.

    • ES6 adds new array iteration syntax: for (var x of array) which asks for an iterator object of the thing to be iterated over, and then the loop calls that iterator’s next() method once for each loop iteration, returning the successful value

Chapter 4: Mixing (Up) “Class” Objects

  • This chapter spends a lot of time going over OO concepts
  • JS has class-like syntactic elements such as new, instanceof, and ES6 class but it does not actually have classes
  • Faking classes in JS is generally not worth it

Chapter 5: Prototypes

  • JS objects have an internal property denoted as prototype, which is a reference to another object
  • If a property lookup via property access fails, the prototype object is followed until is it empty or the property has been found. If we reached the end of the prototype chain and we still haven’t found the property, undefined is returned
  • Object.prototype is the top level of the normal prototype chain
  • Attempting to set the value of a property on an object that said object doesn’t have, but the property exists further up the prototype chain, that property will be added to the targetted object and will shadow the property up the prototype chain. If the property up the chain has a setter, however, then that setter is called and no shadowing occurs
  • A read-only property up the chain will also prevent said property from being set lower on the chain
  • Function.prototype is a special object that all constructor objects created will have their prototype property pointing towards
  • Foo.prototype.constructor points to the “constructor” function (function called with new in front of it)
  • Object.create will take a function as its argument to set as the returned object’s prototype
  • Assinging properties to this inside the constructor function will add that property onto each object
  • Adding functions directly to the prototype chain will give all functions “constructed” acess to those functions
  • ES6 introduces Object.setPrototypeOf

Chapter 6: Behavior Delegation

  • This chapter describes a different way of thinking of the prototypical nature of JS other than “class” or “inheritance”
  • Instead, think of prototype nature as one object delegating behavior to another via oibject linking & the nature of the getter algorithm to traverse up the prototype chain.
  • The base ‘object’ has utility methods; unique objects are then created with Object.create, passing the base object as the prototype. New objects can then delegate behavior back to base object
  • Object state should be on delegators (objects created from base), not delegates (base object)
  • Use specific method names on base/children to prevent shadowing, which makes it harder to reference a specific variable/function
  • Two objects can not be linked to each other; this would create an infinite loop for the Getter algorithm as it attempts to traverse the prototype chain
  • ES6 allows concise method declarations in an object literal, making for nicer-looking syntax when declaring functions as part of an object)
    • Concise method declarations creates annonymous function expressions instead of named function declarations so watch out when debugging!
    • An implementation-specific detail is to set the internal name property of the created function to have an appropraite name so the concise function still shows up in stack traces with a name
  • Using instanceof and prototype for type introspection is hacky and kind of sucks - try to solve problem another way if possible
  • Another way to do type introspection is to check if a property exists (eg: is this a Promise? If it is, it should have the then property method on it)
  • One can also use isPrototypeOf and Object.getPrototype to do these checks with OLOO style (objects linked to other objects)

Appendix A: ES6 class

  • extends automatically creates the prototype link between “classes”
  • super allows one to refer to a method of the same name one level up the chain
  • ES6 has no support for class properties, only methods
  • class is just syntactic sugar over Prototype mechanisms. It does not copy definitions statically at declaration time as in traditional OO. It uses prototype delegation instead
  • super is not bound at call time but at creation (of “class”) time.

Types & Grammar

Chapter 1: Types

  • ECMAScript types:
    • Undefined
    • Null
    • Boolean
    • String
    • Number
    • Object
    • Symbol (added in ES6)
  • NaN is actually a property on the global object, or Number.NaN - these have the same value
  • Use typeof operator to discen the type of a variable / literal (except null which is buggy with typeof)
    • typeof also lets you know if something is a function
  • Variables don’t have types; values do
  • Undefined != Undeclared since variables dont have types. A variable having a type undefined does not mean it is not declared

Chapter 2: Values

  • Strings are immutable while arrays are not; while strings may seem like arrays of characters due to sharing some methods with arrays, they are not. Also, the index operator on strings was not always valid JS
  • Since strings are immutable, methods that would “alter” them actually return a new string
  • Reversing a string by converting it to an array, reversing it and then joining the array back into a string won’t work for strings with complex unicode characters
  • All numbers in JS are double percisions floating points (64-bit) (IEEE 754 spec)
  • Numbers can be expressed in base-10, hex (0xff), octal (0o363), binary (0b1101)
  • IEEE 754 has a bug where 0.1 + 0.2 !== 0.3. JS has issues with very small numbers
    • Get around this with ES6’s Number.EPSILON which is a rounding tolerance value. If the absolute value of n1 - n2 is less than EPSILON, it’s close enough to be equal
  • Number.MAX_VALUE is misleading; the biggest int that can be represented is just over 9 quadrillion
    • This is defined in ES6 as Number.MAX_SAFE_INTEGER. There is also a Number.MIN_SAFE_INTEGER
  • ES6 lets us test if a number is an int with Number.isInteger. Before that, you’d check if the number modulus 1 would equal 0. A fraction would not
  • Bitwise operators only work on 32-bit numbers
  • void N makes N return undefined, but doesn’t actually alter N’s contents
  • Use Number.isNaN to check if the value of an operation/variable is Not a Number
  • JavaScript can also return Infinity (divide by 0) and -0
    • Infinity can also be returned if an operation results in a value that is too large and IEEE 754’s rounding mode decides to round up to Infinity (???)
  • Object.is can be used to check definitively if something is NaN or -0
  • References in JS always point to a value, not at each other
  • The type of value controls whether said value is assigned by value-copy or reference-copy
    • Scalar primitives are always assigned/passed by value-copy: null, unefined, string, number, boolean and symbol
    • Compound values always create a copy of the reference on assignment or passing: objects, arrays, boxed object wrappers and functions
  • Passing an array by value copy, you’d need to manually make a copy & pass it (ie: array.slice())

Chapter 3: Natives

  • Internal Class property is returned by Object.prototype.toString. Tells you what “type” of native (String, Number, RegEx, Symbols, Boolean, etc)
  • Primitives are auto-boxed by their respective object wrappers
  • Unbox values with valueOf()
  • Avoid using native constructors and use the literal forms & let auto-boxing happen
    • Using RegExp can be useful to dynamically building patterns for a RE
  • Date() and Error() are useful since they have no literal forms.
  • Symbols are unique values that can be used as properties on objects. Use the Symbol() native to create your own
  • Prototypes for natives (eg: Number.prototype, String.prototype, etc) contain the behaviour unique to that subtype
  • Array, Function and RegExp’s prototypes are actually empty instances of themselves, making them good for ‘setting defaults’

Chapter 4: Coercion

  • Type casting is converting a value from one type to another explicitly, coercion is that process being done implicitly
  • JS coercion always results in a scalar primitive
  • Internal “abstract operations” handle how values become a certain type. They are defined in the spec and called ToString, ToNumber and ToBoolean. ToPrimtive also exists
  • Can’t JSON.stringify objects with circular references. undefined won’t be stringified either
  • Falsy values in JS: undefined, null, false, 0, NaN, ""
  • Explicit coercion with string/numbers using natives w/o the new keyword. +num is a hack to convert num -> str. That trick also converts Date -> Number (unix time)
  • The ~ operator (bitwise NOT) can be used to coerce -1 to 0 (falsy) and everything else to truthy… uh ok?
    • Also, it’s the math. concept of two’s complement. I remember once knowing what that was
  • Coerce string -> number with parseInt / parseFloat
  • Coerce to bool with the !! prefix operator (double !)
  • ToPrimtive abstract operation will invoke an object’s toValue function when attempting to do coercion
  • What gets returned when using || or &&?

    a || b;
    // roughly equivalent to:
    a ? a : b;
    
    a && b;
    // roughly equivalent to:
    a ? b : a;
    
  • From what I’ve gathered from this chapter, there are way too many rules for implicit coercion for it to be worth using. The author suggests its worth it sometimes due to sytactic cleanliness but if I need to memorize a huge amount of rules to use implicit coercion safely, I’ll pass.

Chapter 5: Grammar

  • “Assignment expressions”
  • All statements return a value, even if its just undefined
    • No way to safely capture this value at the moment, ES7 may add a do expression
  • Expressions usually don’t have an effect, but some may (like function calls) and some definitely do (increment/decrement operators)
  • , operator allows you to string multiple expressions into a single statement
  • What’s happening here?

    [] + {}; // "[object Object]"
    {} + []; // 0
    

    The first line: [] is coerced to "" and {} is interpreted as an empty object and is coerced to a string.

    The second line: {} is an empty block. + [] after it is coercing an empty array to a number which will be 0

  • This one is crazy: JavaScript doesn’t have an else if statement. It’s just legal to have if and else omit the {} around their block if it contains a single statement

  • try, catch AND finally!

  • There are two classifications of JS errors: early (compiler thrown, uncatchable) errors and “runtime” (try/catchable)

Appendix A: Mixed Environment JavaScript

  • Creating DOM elements with an id attribute creates global variables of those same names
  • Never extend native prototypes
  • Multiple <script> tags share the same global object, but scope hoisting does not occur across the tags

Async & Performance

Chapter 1: Asynchrony: Now & Later

  • In some browsers, console is async due to I/O being slow & blocking
  • Across all hosting envrionments (browsers, Node) the common thread of JS is the event loop. Until recently, the event loop is a part of the hosting environment. It is a queue of events to process. The JS engine will tell the hosting environment about functions to call back. The browser will insert that function in the event loop when ready
    • One tick of event loop = one iteration
  • setTimeout doesn’t place the callback directly in the event loop; it sets up a timer and when that timer expires, the environment places the callback in the event loop
  • ES6 makes specifications to the event loop which makes it also responsibility of engine, not just host environment. This is due to ES6 promises needing direct control of scheduling operations
  • JS is single threaded
  • JS features run-to-completion behaviour: events are atomic so once a function starts running, the entirety of its code will finish before any other events can
  • However, concurrency allows two or more chains of events to interleave over time so they appaer to be happening at the same time

Chapter 2: Callbacks

All our examples [..] used the function as the individual, indivisible unit of operations, whereby inside the function, statements run in predictable order (above the compiler level!), but at the function-ordering level, events (aka async function invocations) can happen in a variety of orders.

In all these cases, the function is acting as a “callback,” because it serves as the target for the event loop to “call back into” the program, whenever that item in the queue is processed.

  • Callbacks are the most common async pattern
  • No multitasking (parallelism), but fast context switching between different tasks, making small progress on each task in chunks
  • Third party libraries that take callback functions creates an inversion of control in which many things can go wrong and you would have to account for all of them.
  • Various patterns have been developed to try to mitigate issues with callbacks, such as taking success/error callbacks, or the “error-first” callback method.

Chapter 3: Promises

  • Passing a callback to another party for them to invoke it correctly creates inversion of control. Promises fixes this by letting us decide to do when the task finishes or fails
  • “So before I show the Promise code, I want to fully explain what a Promise really is conceptually.” This attitude is the major problem with this series of books. Maybe it’s just my learning style but I’d rather see the code so I can have something to refer to when having all these metaphors thrown at me
  • Promises revolve around the idea of a future value that will be fulfilled successfully or end in failure
  • Promise.all takes an array of Promises and returns a new Promise that waits on all of them to finish
  • You can recognize whether a value is a genuine promise if it has a then method
  • Promises cannot be called too early because the function passed to then is always called async, even if the Promise was already resolved
  • Same reasoning for why it cannot be called too late; callbacks are scheduled when resolve or reject are called by the Promise
  • Nothing will prevent a Promise’s reject or resolve callbacks - not even a JS error
  • Promise.race will return the result of the first promise that finishes
  • Promises can only be rejected/resolved once, so no need to worry about a Promise being called too many times
  • reject and resolve can only accept one argument
  • Exceptions will cause a Promise to be automatically rejected
  • Promise.resolve(immediate, non-thenable value) can be used to get a Promise that is fulfilled with that value
  • Calling then() on a Promise creates & returns a new Promise

Chapter 4: Generators

  • ES6 generators allow us to express async flow using sync-looking code
  • Generators break run-to-completion behaviour functions usually have
  • Generator functions have the following syntax: function* foo() { ... }
  • Construct an iterator from the function: var it = foo() - this does not start an iteration yet
  • Call the first/next iteration: it.next()
  • yield statements pause the generator until the next call of it.next()
  • yield can even be put mid-statement, where the next next(val) call will pass val to the point where yield is in the code

    function *foo(x) {
      var y = x * (yield);
      return y;
    }
    
    var it = foo( 6 );
    
    // start `foo(..)`
    it.next();
    
    var res = it.next( 7 );
    
    res.value;      // 42
    
  • Values can be yielded out of the generator by passing a value after yield. Access it via var res = it.next(); res.value

Chapter 5: Program Performance

  • Run process-intensive tasks in web workers
    • JS still doesn’t have multi-threaded support; Web Workers is a part of the host environment, not the JS language
    • Browser provides multiple instances of the JS engine, each with its own thread
    • One can send messages to worker & listen to messages sent from worker
    • The worker has no access to main program’s resources or the DOM
    • It does have acess to Ajax, sockets, navigator, location, JSON and applicationCachce. You can also load extra JS scripts in with a blocking method called importScripts
    • SharedWorker can be used to create a Worker for all instances/tabs of your page
  • Single Instruction, Multiple Data (SIMD) allows for data parallelism as the same suggests
    • Will be useful for WebGL
  • asm.js is writing your programs in such a way that the engine uses crazy optimizations.

Chapter 6: Benchmarking & Tuning

  • Use console.time(label) / console.timeEnd(label if possible

ES6 & Beyond

Chapter 2: Syntax

  • var binds variable to enclosing function/global, let will bind to the current block
  • Functions declarations inside of a block are now scoped to that block
  • Parameters can have default values now in ES6
  • Concise object methods x () { }, have access to using super on an object
  • Arrow functions are annonymous function expressions allow the contents of the arrow function to have the same this as the function it was defined in. In other words, the binding of this becomes lexical rather than dynamic
  • ES6 intoduces for (var val of a) iteration syntax with objects with iterables
  • A new primtiive, Symbols, were added. Symbols have a unique value. There is no symbol literal, you must use Symbol() to create them
    • Symbol.for looks in the global symbol registry rather than a local one

Chapter 5: Collections

  • TypedArrays were added to add structured access to binary data using array-like semantics
    • First, create an ArrayBuffer
    • Then create a new “view” with a TypedArray using that array buffer as a way to define how to structure the binary data held in the array
    • A single buffer can have multiple views
  • Maps are key-value pairs where the key can be an object
  • Sets are unique lists of values
  • WeakMaps are maps with the key being weakly held, so the GC can clean up/remove that entry if its the last reference to an object

Chapter 6: API Additions

  • Array.of is now preferred function-form constructor for Arrays. Takes an argument of what the length should be
  • Object.assign is a part of ES6 - older browsers may not support it

Node

Catching errors in Node.js

process.on('uncaughtException', () => {
  // do stuff like cleanup & die/restart
});

GraphQL

Queries

GraphQL queries asks for fields on objects. For example:

{
  hero {
    name
  }
}

returns:

{
  "data": {
    "hero": {
      "name": "R2-D2"
    }
  }
}

Our query can also contain nested fields, allowing us to fetch lots of data at once:

{
  hero {
    name
# Queries can have comments!
    friends {
      name
    }
  }
}

Queries are usually dynamic so being able to pass arguments to a query is useful:

{
  human(id: "1000") {
    name
    height
  }
}

In the above query, we are searching for a human object with an id of “1000”. Nested fields such as height can take arguments as well:

{
  human(id: "1000") {
    name
    height(unit: FOOT)
  }
}

Arguments can be of any of the default types shipped with GraphQL, or a custom type defined by a developer.

If you want to query for the same field with different arguments, aliases are required. This is because the returned data field names match the field names given in the query. For example:

{
  empireHero: hero(episode: EMPIRE) {
    name
  }
  jediHero: hero(episode: JEDI) {
    name
  }
}

will return data with the field names matching the alias names:

{
  "data": {
    "empireHero": {
      "name": "Luke Skywalker"
    },
    "jediHero": {
      "name": "R2-D2"
    }
  }
}

Queries involving many repeating fields (e.g.: comparing two Character types) can be aided by the use of fragments. A fragment is a set of fields that can be included in different queries:

# our query
{
  leftComparison: hero(episode: EMPIRE) {
    ...comparisonFields
  }
  rightComparison: hero(episode: JEDI) {
    ...comparisonFields
  }
}

# defining our fragment
fragment comparisonFields on Character {
  name
  appearsIn
  friends {
    name
  }
}

To use variables in a query, the variable must be defined in the argument list for the query with a dollar sign ($) prefix. (e.g.: $episode) and given a type (of scalar, enum, or input objects):

query HeroNameAndFriends($episode: Episode) {
  hero(episode: $episode) {
    name
    friends {
      name
    }
  }
}

If the incoming request has an object with a field matching the variable name (e.g.: { "episode": "JEDI"), passing it to our query should fill in the variable and the query will execute.

If the variable definition is suffixed with a !, it means that argument is required. With no ! suffix, the argument is considered optional.

Giving an operation name to a query is useful for debugging and to make the query less ambigious to other developers. For example, in the above code snippet showing off GraphQL variables, the query is given a name of HeroNameAndFriends.

Directives are useful for conditionally including certain fields in a query, allowing us to dynamically change the shape of our query:

query Hero($episode: Episode, $withFriends: Boolean!) {
  hero(episode: $episode) {
    name
    friends @include(if: $withFriends) {
      name
    }
  }
}

The above query Hero will only include the nested object friends if the variable $withFriends has a value of true. There are two directives included in the core GraphQL spec:

  • @include(if: Boolean): only includes field/fragment if argument is true
  • @skip(if: Boolean): skips the field/fragment if argument is true

    An operation that causes a write should be created as a mutation rather than a query:

    mutation CreateReviewForEpisode($ep: Episode!, $review: ReviewInput!) {
      createReview(episode: $ep, review: $review) {
        stars
        commentary
      }
    }
    

    Assume that the following data payload is included in our request:

    {
      "ep": "JEDI",
      "review": {
        "stars": 5,
        "commentary": "This is a great movie!"
      }
    }

    The payload will be substituted into the mutation’s arguments. Multiple mutations can be included in one operation. However, unlike queries which are executed in parallel, mutations are executed sequentially, one after another.

    If a field returns an interface or union type, using inline fragments are required to access the appropriate fields, depending on the return type:

    query HeroForEpisode($ep: Episode!) {
      hero(episode: $ep) {
        name
        ... on Droid {
          primaryFunction
        }
        ... on Human {
          height
        }
      }
    }
    

    The return type Character can be a Human or a Droid depending on the $ep argument. In the direct selection, we can only choose fields directly on Character. To access fields on specific types, we use ... on TYPE syntax to create an inline fragment.

    If one wants to get the name of the object typed at any point in the query, use the __typename metafield anywhere in a query.

React

General

  • React elements are plain objects
  • Render with ReactDOM.render(react_element, target_element). reat_element will be a child of target_element.
  • Components can be JS functions that take props object argument and returns a React element, or an ES6 class inheriting from React.Component

State & Lifecycle

  • State requires ES6 class components, not functional components
  • Set this.state in the constructor of the component. Don’t forget to call super(props) on the first line of the constructor+. At least render() must be defined.
    • Calling super(props) in the constructor guarantees this.props are not undefined later in the constructor
  • Only modify state with this.setState({prop: 'value'}, callback)
  • State updates can be async - React may batch setState() calls for performance. Don’t rely on their values to calculate the next frame state
  • When an instance of a component is being created/inserted into the DOM (“mounted”):
    1. constructor()
    2. componentWillMount()
      • When doing SSR, this is the only lifecycle method called. Use constructor instead though
    3. render()
    4. componentDidMount()
  • When props/state are changed, the component is updated & re-rendered:
    1. componentWillReceieveProps(nextProps)
    2. shouldComponentUpdate(nextProps, nextState)
      • Where we decide if component should re-render or not.
      • Does not prevent children from updating when their state changes
      • If false is returned, next lifecycle methods are not called
    3. componentWillUpdate(nextProps, nextState)
    4. render()
    5. componentDidUpdate()
  • When the component is being removed from the DOM:
    1. componentWillUnmount()
  • defaultProps and propTypes are properties that can be used to provide default property values and validate prop types in development mode

Events

  • Keep in mind that class methods in JS are not bound to said class by default. You have to either use experimental arrow syntax to define a function expression in the class, an arrow function in the event callback (although this creates a new function on every render - can cause needless child re-renders!), or bind the method to the class in the constructor.

Forms

  • Forms work differently than other DOM elements as forms keep some sort of internal state
  • Controlled components: form (or React Component) state’s single source of truth is React state. Set child DOM element’s values according to component’s internal state, and change state when altering form controls, causing re-renders
  • textarea can use a value property instead of defining its contents via children nodes.
  • For select / option, the controlled component is select
  • Uncontrolled components use ref to values from the DOM instead of using React state
  • Use the defaultValue attribute if you want to set an initial value but you don’t want that value to be controlled by React

Composition of Components

  • Use composition over inheritence
  • Access child nodes passed to React elements with props.children
  • “Specialization” pattern: A more specific component renders a more generic one and configures the props passed down to generic.

ref

  • Special attribute that takes a callback which is executed right after the component is mounted/unmounted. If attribute is on an HTML element, the callback argument is the DOM node.
  • Legacy ref via string is discouraged and will probably be removed in a future release

Testing Components

There are a few different methods of testing components:

  • Smoke test: Testing that a component renders without throwing an error
  • Shallow rendering: Testing that a component has specific output
  • Full rendering: Tests component output, lifecycle and state changes

Performance

  • Make sure you are using the production build of React
  • Load app with ?react_perf in the query string to be able to use Chrome DevTools profiler to see performance on components, grouped under ‘UserTiming’ label
  • Override shouldComponentUpdate() (which returns ‘true’ by default) for further fine-tuning of specific components.
  • Inherit a component from React.PureComponent to have shouldComponentUpdate() do a shallow comparison of all props/state values to see whether to perform an update or not
    • With complex objects, get around this by never directy mutating the object & always returning new objects, so shallow comparison of objects will fail

Reconciliation

  • render() returns a tree of React elements based on the current values of props & states in the component tree
  • Diff algorithm works on two assumptions:
    • Two elements of diff. types produce diff. trees
    • Devs can hint at which child elements are stable across renders with the key prop
  • Diff algorithm first compares two root elements (of current/next tree)
    • If elements are of diff. types, the entire old tree is destroyed (components receieve componentWillUnmount()) and a new tree is built up with those respective lifecycle methods being called
    • If the HTML elements are of the same type, React looks at the attributes and keeps the underlying DOM node while updating changes attributes. It then recurses on child nodes.
    • When iterating over children, React iterates over both lists (current tree & new tree) at the same time and generates a mutation when reaching a difference.
      • This can lead to problems if the first element of a list was changed, causing the rest of the list to re-render. Use the key attribute to get around this by letting React know which elements are actually new and which are not

Context

  • A way to ‘cheat’ the unidirectinal flow of React.
  • Unstable & can break in future releases; use Redux or MobX instead to manage state
  • Component that wants to subscribe to context changes needs contextTypes property on component defined
  • Component that wants to push context data down the tree need the getChildContext() method defined and the childContextTypes property defined on the component
  • If contextTypes is defined, the following lifecycle methods receieve an additional parameter:
    • constructor
    • componentWillReceieveProps
    • shouldComponentUpdate
    • componentWillUpdate
    • componentDidUpdate
  • Do not upodate context. It’s broken according to React devs.

Higher-order Components

  • A higher-order component is a function that takes a component and returns a new component; it transforms one component into another
    • eg: Redux’s connect method which takes a component & a map of reducers to subscribe to and returns the connected component
  • Don’t mutate the orignal components: use composition to wrap original component & render/send props to it in the desired fashion
  • Set a HOC’s displayName to be the HOC wrapped in the original component name for easy debugging:

    function withSubscription(WrappedComponent) {
      class WithSubscription extends React.Component {/* ... */}
      WithSubscription.displayName = `WithSubscription(${getDisplayName(WrappedComponent)})`;
      return WithSubscription;
    }
    
  • Don’t apply a HOC in render(); it will return a new element on each render. Apply HOC outside of component definitions so the resulting component is only created once

Interesting Articles

Defense Against the Dark Arts: CSRF Attacks

  • Initial HTTP requests do not know how to send custom HTTP headers (e.g.: for including a token being stored in localStorage)
  • Cookies open apps up to CSRF attacks (Cross-Site Request Forgery)

    For example, if someone included the following img tag on their site:

    <img src="http://hogwarts.edu/mcgonagall/send-message?to=all_students&msg=Potter_sux">

    The browser will send a HTTP GET to that URL, sending along any cookies for that particular domain. This means a malicious user can send requets on behalf of anyone who visits the page with that img tag on it.

    This particular exploit can be remedied by changing the /send-message route to only accept HTTP POST requests.

  • Unfortunately, HTTP POST can also used in a CSRF attack with the following:

    <form method="POST" action="http://hogwarts.edu/mcgonagall/send-message?to=all_students&msg=Potter_sux">

    This, coupled with JavaScript that executes the form when the page loads, will cause the malicious user to act on behalf of the visiting user. Neither HTTPS nor secure cookies will fix this.

  • One way to protect against CSRF attacks via HTTP POST is by enforcing a same-origin policy. Browsers will include either (or possibly both) of the HTTP headers Referer and Origin. If these headers do not match the site we expect them to be coming from (e.g.: http://hogwarts.edu/), we can reject them as malicious requests.

  • Another method is to use Custom Request Headers. HTTP requests send by browser elements such as form or img don’t need to obey the same-origin policy, but they cannot send custom headers in their requests. By setting a header that a browser element request would not (e.g.: Content-Type), we can be sure that requests with this header are XHR requests. This check should be used in addition to same-origin, not as a replacement.

How to deploy Django with Docker

  • Use a docker-compose.yml file to define & start/stop environments
    • Essential services: Django, Postgres, Caddy as a HTTP proxy
  • The command for the container with Django may need to wait for Postgres to be up with the following command:

    bash -c "while ! nc -w 1 -z db 5432; do sleep 0.1; done; ./manage.py migrate; while :; do ./manage.py runserver_plus 0.0.0.0:8000; sleep 1; done"
  • Run manage.py commands with the following command:

    docker-compose run web /code/manage.py run
  • Article contains a Dockerfile, startup script and more that would be useful for deploying Docker to production

Native Apps are Doomed

(article was posted 2016-11-07)

This paragraph in the intro is what got me interested in this subject at all:

This is where it gets interesting. Just like any native app, the progressive web app will have its own home screen icon, and when you click on it, the app will launch without the browser chrome. That means no URL bar and no web navigation UI. Just the phone’s usual status bar and your app in all its almost-full-screen glory.

So it’s a web app that acts like a native app? If we can leverage hardware APIs here, we could be laughing (in terms of useful stuff for kpd-i).

PWAs have specific requirements such as:

  • Must be HTTPS
  • Valid web manifest with required props.
  • Must have a service worker
  • Manifest’s `start_url` must always load, even when offline
  • Resopnsive design (obviously!)

    It’s not very clear but from this, it seems like gestures are not supported:

    What many app builders seem to forget is that if you build a progressive web application, you must be able to navigate the application without the browser chrome and browser navigation gestures. The mobile devices assume that you’ve built your own navigation into the app.

    For example, if you have an about page, that page must have a link back to the app UI, or the users will have to close and reopen the app to get back to your main app UI.

    For iOS, web manifests need a `meta` tag in `header` to be recognized:

    <meta name="apple-mobile-web-app-capable" content="yes">

    What is a service worker:

    Service Worker is a recent web platform specification that allows you to cache resources locally in order to make sure that your app still works, even if the user is not connected to the internet.

    It works by hijacking your network requests and serving responses from a local cache when the user is offline.

    This seems very interesting. I’d like to use this in future apps of my own.

    The only downside vs. native apps at the moment is lack of hardware support

    Here is a Cordova plugin that supports service workers!

    Article’s conclusion:

    In the short term, you may still want to produce a hybrid app that can take advantage of some device APIs that are still not available using web standards.

    My takeaways:

    • Learn about Service workers! They seem neat.