adamkdean

software engineering

Mount Docker volume locally

By Adam K Dean on

Here is something that's cool.

You can have a docker volume, docker volume create example, and you can mount this volume to a container, docker run -v example:/path image. You can also mount local directories to containers, docker run -v $(pwd)/local:/path image. But something you can't do, is mount a local directory to volume.

Scenario: you have a volume, you have a container using that volume, and you have some software that needs to access that volume data locally, outside of docker.

🚫docker run -v $(pwd)/local:example image isn't going to work.

Nor can you mount two things to the same place.

🚫docker run -v example:/example -v $(pwd)/local:/example image isn't going to work either.

The solution is to use intermediate symlinked container.

I call this solution the map volume.

#
# map volume
#
FROM busybox
RUN mkdir /volume
RUN ln -s /local /volume
CMD tail -f /dev/null

In this dockerfile you can see that we create a directory (later to be used as a mount point), and then we create a symlink directory pointing to the same place.

$ docker build -t mapvolume .
$ docker volume create my_precious_data
$ docker run -d -v my_precious_data:/volume -v $(pwd)/my_precious_data:/local mapvolume

We create a volume my_precious_data, then we run the mapvolume, first mounting the volume my_precious_data to the container path /volume, and then mounting the local directory $(pwd)/my_precious_data to the container path /local which as we saw, is actually /volume.

Now you have mounted a docker volume to your local filesystem.

Update: I realised today that the original dockerfile wouldn't stay alive. I therefore changed the CMD to CMD tail -f /dev/null to keep the container alive. Thanks to bigdatums.net for that snippet.

DADI Web & Browserify

By Adam K Dean on

Earlier tonight I wrote about DADI Web & SCSS using DADI Web middleware. Well, it gets even better. The following JavaScript preprocessor (based on node-enchilada) will serve up your JavaScript files all wrapped up using Browserify. That means you can use CommonJS and keep your client-side JavaScript tidy.

Again, all you need to do is drop this file into your workspace/middleware directory and make sure the paths are correct.

const enchilada = require('enchilada')
const path = require('path')
const workdir = process.cwd()

const Middleware = function (app) {
  app.use(enchilada({
    src: path.join(workdir, 'workspace', 'public'),
    cache: process.env.NODE_ENV !== 'development',
    compress: process.env.NODE_ENV !== 'development'
  }))
}

module.exports = function (app) {
  return new Middleware(app)
}

module.exports.Middleware = Middleware

Protip: enchilada looks inside the src directory for a subsequent js directory.

Learn more about DADI Web.

DADI Web & SCSS

By Adam K Dean on

One of the many great things about DADI Web is how easy it is to plug in middleware, such as the following SCSS preprocessor (based on node-sass-middleware) which takes any request with prefix, in this case /css, and compiles it's counterpart from the src directory into the dest directory before serving it up.

All you need to do is drop this file into your workspace/middleware directory and make sure the paths are correct.

const sassMiddleware = require('node-sass-middleware')
const path = require('path')
const workdir = process.cwd()

const Middleware = function (app) {
  app.use(sassMiddleware({
    src: path.join(workdir, 'workspace', 'public', 'scss'),
    dest: path.join(workdir, 'workspace', 'public', 'css'),
    prefix: '/css',
    outputStyle: 'compressed',
    debug: process.env.NODE_ENV === 'development'
  }))
}

module.exports = function (app) {
  return new Middleware(app)
}

module.exports.Middleware = Middleware

Learn more about DADI Web.

Pass Request buffer through Redis

By Adam K Dean on

In a follow up to my post last night, the following code takes an Express request, performs an external request, stores that response in a buffer which it then stores in Redis, which it then reads back, converts back into a buffer and sends that back to the Express response.

This doesn't store the headers or status code in Redis but it could easily be done.

This allows playback of a request/response cycle. Interesting concept really.

'use strict'

const express = require('express')
const request = require('request')
const streamBuffers = require('stream-buffers')
const through2 = require('through2')
const redis = require('redis')

const client = redis.createClient()
client.on('connect', () => { console.log('redis connected' )})
client.on('ready', () => { console.log('redis ready' )})
client.on('reconnecting', () => { console.log('redis reconnecting' )})
client.on('error', () => { console.log('redis error' )})
client.on('end', () => { console.log('redis end' )})

const app = express()
app.use('/', (incomingRequest, outgoingResponse) => {
  const outgoingRequest = request('http://via.placeholder.com/800x600?text=example')
  const randomKey = Math.random().toString(36).substr(2)
  const stream = new streamBuffers.WritableStreamBuffer()

  let _statusCode = null
  let _headers = null

  outgoingRequest.pipe(through2(function (chunk, enc, callback) {
    stream.write(chunk)
    callback()
  }))

  outgoingRequest.on('end', () => {
    const contents = stream.getContents()
    const base64 = contents.toString('base64')
    client.set(randomKey, base64)
    client.get(randomKey, (err, reply) => {
      const newBuffer = new Buffer(reply, 'base64')
      outgoingResponse.set(_headers).status(_statusCode)
      outgoingResponse.end(newBuffer)
      client.del(randomKey)
    })
  })

  outgoingRequest.on('response', (incomingResponse) => {
    _statusCode = incomingResponse.statusCode
    _headers = incomingResponse.headers
  })
})

app.listen(8000, () => {
  console.log('listening on 8000')
})

Pipe Stream to Express

By Adam K Dean on

As part of a project I'm working on, I need to grab some data via HTTP/S and transmit it as binary, but with access to it's headers. The following is a quick proof of concept to listen for HTTP requests with Express, request an external image on request, and pipe the response back through to the Express response socket while having access to response metadata such as headers.

'use strict'

const express = require('express')
const request = require('request')
const through2 = require('through2')

const app = express()
const imageUrl = 'http://via.placeholder.com/800x600?text=example'

app.use((incomingRequest, outgoingResponse) => {
  const outgoingRequest = request(imageUrl)
  const bufferOnPipe = through2(function (chunk, enc, callback) {
    this.push(chunk)
    callback()
  })
  const bufferedResponse = outgoingRequest.pipe(bufferOnPipe)

  outgoingRequest.on('response', (incomingResponse) => {
    if (incomingResponse.statusCode === 200) {
      console.log('statusCode:', incomingResponse.statusCode)
      console.log('headers:', incomingResponse.headers)
      bufferedResponse.pipe(outgoingResponse)
    } else {
      console.log('non-200 statusCode:', incomingResponse.statusCode)
    }
  })
})

app.listen(8000, () => {
  console.log('listening on 8000')
})