Mokum NIB (New Image Backend)

Published by @squadette on 2016-12-13

Mokum NIB (New Image Backend)

User-visible requirements:

  • deliver images resized exactly to the visible size, saving bandwidth and latency;
  • allow uploading images in non-baseline but still useful formats, such as TIFF, WEBP, BMP, etc. Uploaded images need to be converted to baseline format (JPEG or PNG);
  • normalize EXIF rotation;
  • strip EXIF metadata;
  • allow “cover images”, such as previews for longcat-format images, and first frame of animated GIFs;
  • fix iCCP colorspace to fix “too dark photos” (https://mokum.place/mokum-support/21876);

Technical requirements:

  • backup-first approach;
  • allow multiple image delivery servers (mostly to handle current disk space issue);
  • gracefully disallow hotlinking;
  • do not resize animated GIF files: “the only winning move is not to play”;

Proposed architecture

Images uploaded by user are Stage0. Stage0 images are stored in S3 for backup.

Stage0 images are converted to Stage1 images once after uploading. Stage1 images have the same number of pixels as Stage0, but are normalized:

  • convert to JPEG or PNG, if needed;
  • normalize EXIF rotation;
  • strip EXIF metadata;
  • fix colorspace;
  • optimize JPEG with Mozjpeg to save storage.

For some images it is possible that stage1 is exactly the same as stage0.

There could be more than one Stage1 image for one Stage0 image, for example GIF file cover frame and the GIF file itself.

Stage1 images are stored on a filesystem, accessible to resizing HTTP server.

Stage2 images are created from Stage1 images on the fly by resizing HTTP server. Stage2 image URLs contain explicit width, height, DPR (device pixel ration) and “intent” (thumbnail or full-size). Also, Save-Data: on HTTP header is handled here. Generated image files are cached by standard nginx caching proxy.

Resizing server requirements

Resizing server converts Stage1 images to Stage 2 images by resizing.

I. Stage1 images are stored on a filesystem with filenames in the following format:

/var/storage/012/345/678/mokum-{UUID}.(jpg|png)

“/var/storage/” part is specified on command line on server start. “012/345/678” part is insecure image ID. “{UUID}” is secure part of image filename, in UUID v4 format.

Other formats, except for jpg and png, are not allowed.

II. Resizing HTTP server accepts HTTP GET requests from nginx proxy in the following format:

GET /012/345/678/mokum-{UUID}-{WxHxC}.(jpg|png) HTTP/1.1

“012/345/678” and “{UUID}” are used to find the filename for resizing.

“{WxHxQ}” is width, height and JPEG quality of requested image, e.g. “200x300x95”. For PNG files quality is ignored.

Width and height of <50 or >2048 pixels (configurable by changing sources) is not allowed. Enlarging images is not allowed. Server should return either 400 Bad Request or original image in that case.

Quality <5 or >100 is not allowed and reset to predefined value of 90.

Requests not matching this format must return 404. Requests for missing files must return 404.

III. Compressed images are returned in HTTP response. Caching, access control, prevention of parallel generation of the same asset, etc. will be implemented in Nginx.

Let us repeat again that Stage 1 are already preprocessed and normalized, so resizing server does not need to concern itself with anything except for resizing.

Implementation of image resizing itself has a potential for optimizations. For now “default/easiest” implementation will probably suffice. Eventually we’ll probably want to experiment with:

  • sharpening of thumbnail images, Flickr-style;
  • optimizing JPEG output with Mozjpeg-based code;

Please share your feedback if you have any questions or concerns about this specification.

As of now it seems that this implementation: https://github.com/pierrre/imageserver is capable of everything that we need. It is implemented in Go, which has a nice property of easy deploy. It is claimed to be fully modular, but examples don’t do what we need, so we need an implementation with modules correctly arranged and tested.

However, we’re open for suggestions. Commercial alternatives include https://www.imgix.com/, but for 150k images we estimate around $300/month which is way too much.

Thank you,


2015-2017 Mokum.place