Cyberponies: Django and the machines

Presenter Notes

Machines?

  • Thousands of remote devices
  • Full of sensors
  • Remote command, local interactions
autolib_chargepoint.jpg

Presenter Notes

What do we need?

  • Manage devices
    • Inventory
    • Register new devices
    • Push upgrades

Presenter Notes

What do we need?

  • Manage devices
    • Inventory
    • Register new devices
    • Push upgrades
  • Monitor
    • Record events/updates from devices
    • Detect issues, lost devices, ...

Presenter Notes

What do we need?

  • Manage devices
    • Inventory
    • Register new devices
    • Push upgrades
  • Monitor
    • Record events/updates from devices
    • Detect issues, lost devices, ...
  • Actually use the devices
    • Aggregate information for end users
    • Transform user requests into commands

Presenter Notes

Inventory

  • Seldom changes
  • Lots of data: model, manufacturer, serial number, mac addresses, ...
  • Including components and subcomponents

Presenter Notes

Inventory

class KioskModel(models.Model):
    """A version of our kiosks."""
    reference = models.SlugField(max_length=16)
    commercial_name = models.CharField(max_length=32)

    manufacturer = models.ForeignKey(Manufacturer)
    family = models.ForeignKey(KioskFamily)

Presenter Notes

Inventory

class KioskModel(models.Model):
    """A version of our kiosks."""
    reference = models.SlugField(max_length=16)
    commercial_name = models.CharField(max_length=32)

    manufacturer = models.ForeignKey(Manufacturer)
    family = models.ForeignKey(KioskFamily)

class Kiosk(models.Model):
    """One specific kiosk."""
    kiosk_model = models.ForeignKey(KioskModel)
    firmware_version = models.ForeignKey(FirmwareVersion)

    customer = models.ForeignKey(Customer)
    location = models.ForeignKey(InstallationSite)
    notes = models.TextField()

Presenter Notes

Inventory: Django benefits

Get all the Django goodness:

  • Simple admin interface
  • New fields? django.db.migrations helps a lot
  • Django apps for history tracking, analysis, reporting...
  • Use APIs (with the excellent django-rest-framework) for automatic inventory updates

Presenter Notes

Inventory: remaining problems

Embedded systems specifics:

  • Registering inventory at the factory or by the device?
  • Pushing upgrades

Presenter Notes

Inventory: remaining problems

Embedded systems specifics:

  • Registering inventory at the factory or by the device?
  • Pushing upgrades

Security:

  • Ensuring the inventory update comes from the right device
  • Checking that all data comes unaltered

=> Some problems might be better solved by dedicated tools (salt, ...)

Presenter Notes

Monitoring

Critical for operations:

  • A device crashed, why?
  • Does the latest firmware perform better/worse?
  • The only way to get a view of the real world your devices live in

=> Monitoring will be a huge part of your network traffic.

One update every five minutes, for 3000 devices, means 10 updates per second...

Presenter Notes

Monitoring: Django benefits

  • With PostgreSQL, Django can handle great amounts of data (think 100 000 000 lines!)
  • Beyond, use dedicated storage engines (e.g elastic)
  • Interface through Django to build reports mixing inventory and monitoring data

Presenter Notes

Monitoring: pitfalls

In a distributed system:

  • You will lose messages
  • Some messages might arrive late
  • Don't forget to synchronize devices' clocks

Presenter Notes

Monitoring: pitfalls

Tips

  • Send the full state of the device in each message, even when triggered by a specific event
  • Send the time since last change for simple binary (on/off) sensors
  • Keep a cache of the "last known state" in Django's DB even if the archives are elsewhere

Presenter Notes

Actual logic

Where using Django really helps!

Publish data to users:

  • Maybe aggregated from various devices
  • Multi-devices: users expect the data on a website, in their app, ...

Send commands to devices:

  • On user input
  • In reaction to other events
  • From your operation center

Presenter Notes

Keeping it in Django

Single logic for all requests:

# app/processes.py
def reserve(user, charge_point):
    if charge_point.seen_recently and charge_point.unused:
        user.reservations.create(charge_point=charge_point)
        charge_point.set_color('blue')

Presenter Notes

Keeping it in Django

Single logic for all requests:

# app/processes.py
def reserve(user, charge_point):
    if charge_point.seen_recently and charge_point.unused:
        user.reservations.create(charge_point=charge_point)
        charge_point.set_color('blue')

# app/views/customer.py
def reserve(request):
    # ...
    if form.is_valid():
        processes.reserve(form.customer, form.charge_point)

Presenter Notes

Keeping it in Django

Single logic for all requests:

# app/processes.py
def reserve(user, charge_point):
    if charge_point.seen_recently and charge_point.unused:
        user.reservations.create(charge_point=charge_point)
        charge_point.set_color('blue')

# app/views/customer.py
def reserve(request):
    # ...
    if form.is_valid():
        processes.reserve(form.customer, form.charge_point)

# ws/charge_point.py
def on_card_swipe(charge_point, card):
    if card.is_active():
        processes.reserve(card.owner, charge_point)

Presenter Notes

Challenges

  • Network latency, NATs: you can't directly send a command as part of a Django request
  • "Split brain effect"
    • The datacenter only has a partial view of the world
    • The device itself only knows what its sensors show
    • What if sensors malfunction?

Presenter Notes

Questions?

Presenter Notes

Thanks!

cyberpony.png

Presenter Notes