Padding Blocks with PyCrypto in Google App Engine

I was thinking of a different kind of Zero.

Photo credit: Tora! Tora! Tora!.

Assume (for the sake of argument; no need to tell us why) that one day you find yourself working with Python in Google App Engine, using PyCrypto to encrypt secrets. Unless your plaintexts are always a multiple of 16 bytes in length, you are likely to run into this error:

ValueError: Input strings must be a multiple of 16 in length

The answer is to pad out your plaintext to an appropriate length, but the version of PyCrypto available in App Engine can't do this for you.


PyCrypto is one of the libraries that Google helpfully provides for you in App Engine (though you do need to specify in your app.yaml file that you want it loaded). They do this not merely out of the kindness of their hearts, but because PyCrypto relies on a C extension, and Google does not want you compiling arbitrary C extensions on their servers. This is quite wise of them. It does mean that you are locked in to the version of PyCrypto that Google has chosen to include in App Engine, though, and that version (2.6) happens to be outdated.

A newer version of PyCrypto was released in 2013, and it includes some significant improvements, such as additional block cipher modes and helper functions for handling block padding. If you are using, say, AES with CBC, you must add correct padding to the final block of plaintext in some situations. Sadly, the newer version (2.7a1) is marked as "experimental", and the project is apparently dead. PyCrypto has been forked as PyCryptoDome, which is currently maintained, but which isn't usable in standard App Engine environments.

While it is unfortunate that Google App Engine remains locked to the older version of PyCrypto that lacks the new features, the padding helpers are written cleanly enough that they can easily be included in your project.

Read more…

Your Password Policy Is Wrong

Security fatigue is very real, and puts our data at risk. Promoting effective, human-friendly passphrase policies is a good place to start addressing it.

Earlier this year, I received a marketing email from LifeLock, which linked to a post warning of the ills of weak passwords, and inferring if 'Zuck isn't too cool to have his accounts compromised, neither are we.

Unfortunately, the folks at LifeLock don't appear to be experts in effective password strength policies. I want to address the following statement specifically:

When you develop your new passwords, think long and strong – using upper- and lower-case letters, special characters and numbers. And make sure the resulting passwords aren’t words found in the dictionary.

In the words of Dwight Schrute: WRONG. Adding complexity – by swapping character-case, and injecting numbers and symbols – does not improve the entropy (strength) of a passphrase much. Increasing the length of a passphrase is the best way to improve its entropy, and the easiest way to increase passphrase length is to add random dictionary words.

That's right, the oft-repeated "don't use dictionary words" is woefully inaccurate. In fact, the best method we have for generating high-entropy (very strong) passphrases is based entirely on using a sequence of randomly-selected dictionary words, called Diceware.

Read more…

Google Fiber Is A Death Star

Now witness the firepower of this fully armed and operational battle station wireless antenna.

Photo credit: Star Wars Wikia.

Google Fiber's big announcement last week, that they're going to “pause” the rollout of Google Fiber in new cities, in combination with the resignation of CEO Craig Barratt, led to a lot of speculation that this particular letter in the Alphabet is in trouble. Everyone from Ars Technica to The Washington Post had some fun with this story. So why not us?

We don't want to bore you, so we offer you a different take. Google Fiber isn't in trouble: in fact, it's poised to completely disrupt the ISP market. In short, Google Fiber is the Death Star from Return of the Jedi.

Read more…

Do IoT Botnet DDoS Attacks Threaten The Internet?

The end of the Internets as we know them?

Photo credit: Wikimedia Commons.

The following is an edited transcript of an internal Panda Strike Slack discussion, in which we assess implications of the recent IoT-based DDoS attack and conclude that we need to drink delicious beer.

Dan So the Internets are freaking out today about how people's toasters have become an attack vector. Take this Tweet as a fairly representative example:

Humor aside, isn't this a bit alarmist? After all, the problem was an attack on a specific DNS provider, not the Internet itself, and only those companies using that DNS provider were affected directly.

Read more…

Remote Work: A Love Story

Engagement photo of Jesse and Holly, who are adorable, in Oceanside.

Photo credit: Jessica Harrington.

I’ve worked from home a lot over the past 8 years, for several reasons, but for one vitally important one that isn’t often discussed.

When people talk about working from home they mention something time saving or economical - like saved commute time, money saved eating at home, or the pure joy that comes from getting to wear pajamas all day. The conversation is generally centered around productivity and focus, which is all true and well and good but isn’t the main reason I work from home.

More than being more convenient and productive, working from home helped me maintain my sanity when my fiancee Holly received devastating medical news, launching us into a frightening, painful, and unpredictable journey that we’re still making our way through.

Read more…

Soft Deletes In HTTP APIs

That's okay, delete if it'll make you feel better — we've got you covered! And no time travel necessary. Photo credit: Ervins Strauhmanis .

A fun question came up the other day on our internal Slack channel:

For soft-deletes (where we keep a record in the database, but set a flag that it's been deleted), we're trying to decide between doing that with DELETE <resource> or PUT <resource> with a body {status: 'deleted'}. That way, we can reserve DELETE for hard deletes. Any recommendations?

The argument in favor of the PUT approach is well articulated by Phil Sturgeon:

Delete should probably do what it says, actually delete the item.

On the face of it, what could be more reasonable? But we're going to explain why this heuristic is misguided. Or as I put it in our Slack channel:

He's wrong.

Read more…

Visualizing Distributed Load Tests With JMeter, Elasticsearch, Fluentd, and Kibana

A web site's requests-per-minute recorded by JMeter, stored in Elasticsearch, and visualized with Kibana.

Apache JMeter is a flexible tool for producing load against an application and measuring the results. We used it on a recent project to measure performance of a large and complex web site. The results of the performance tests required aggregation and visualization. The performance tests required a scale-out approach since the desired request rate exceeded the capabilities of a single JMeter node.

JMeter has its own built-in distributed performance testing capability. It also has the capability to visualize the test results. We found both lacking.

Read more…

Version Media Types, Not URLs

A rose by any other name would return a 404.
(Photo credit: Wikimedia.)

Roy Fielding's advice on versioning APIs is, well, succinct:


In a subsequent interview, he thankfully clarifies:

Websites don’t come with version numbers attached because they never need to. Neither should a RESTful API.

In one of our own posts on HTTP, we hinted that HTTP does, in fact, support versioning. After all, as Fielding says in that same interview:

Anticipating change is one of the central themes of REST.

But if you aren't supposed to version APIs, how do you “anticipate change,” exactly? At some point, don't you have to version something?

Read more…

Microservices And Serverless Architecture

Serverless architecture does not imply that there are no servers. It merely means that the unit of deployment is code rather than servers. And it's a perfect fit for worker-based patterns, which we use quite frequently.

One of our favorite patterns at Panda Strike is to have an HTTP API that dispatches jobs to workers. We called this the dispatcher-worker pattern, but its names are exceeded only by its variations. In particular, it's a variation of the microservices pattern.

The basic idea is to ensure that the HTTP API is only concerned with validating requests and dispatching them to the right worker. And, when necessary, relaying a result from a worker back to the client. Meanwhile, each worker need only do one job. For example, one worker might handle registrations, while another handles feed requests.

So you can imagine our excitement about Amazon's support for “serverless” architecture, which happens to fit this pattern perfectly.

Read more…

Design Patterns In HTTP

Christopher Alexander introduced the idea of design patterns in a book about architecture, A Pattern Language. He later extended that approach to a suggestive study of Turkish Carpets A Foreshadowing of 21st Century Art.

We've made the case on this blog that REST is the wrong way to try and understand HTTP. We've also said that it's worthy of study if only because the Web runs over HTTP. We've even contributed a few introductory blog posts to the subject.

But that begs the question—what's the big picture here? Is HTTP's success due to the Web or is it the other way around? Is there some brilliant insight that HTTP captures or is Fielding's dissertation on REST, which he didn't expect anyone to read (pretty typical for a dissertation), the best we can hope for?

Read more…

Older Posts →