How to update docker service using a version from docker-compose

| Comments

Here is a small script which compares the version of the running service and the version in the docker-compose file, if they are different it runs an update.

REDIS_VERSION=$(docker service ls | grep "redis" | awk '{print $5}')
REDIS_NEW_VERSION=$(grep -Po "image:\s*\Kredis:.*" docker-compose.yml)
if [ "$REDIS_VERSION" != "$REDIS_NEW_VERSION" ]; then
    echo "update $REDIS_VERSION -> $REDIS_NEW_VERSION"
    docker service update --image $REDIS_NEW_VERSION app_redis
fi

Env variables for Vue.js in Nginx container

| Comments

The default approach for Vue.js is to use .env files during build stage. But I like more this one with envsubst. There is no need to rebuild image for each environment or when the configuration is changed.

Here is a small modification for entrypoint.sh, so it can replace all variables with VUE_APP_ prefix.

#!/bin/sh

function join_by { local IFS="$1"; shift; echo "$*"; }

# Find vue env vars
vars=$(env | grep VUE_APP_ | awk -F = '{print "$"$1}')
vars=$(join_by ' ' $vars)
echo "Found variables $vars"

for file in /dist/js/app.*;
do
  echo "Processing $file ...";

  # Use the existing JS file as template
  cp $file $file.tmpl
  envsubst "$vars" < $file.tmpl > $file
  rm $file.tmpl
done

exec "$@"

A script to transfer azure storage containers to another storage

| Comments

Here is an example of a script to copy azure storage containers from one storage to another and keep public access level. It uses Azure CLI and AzCopy tools.

#!/bin/bash
STORAGE_NAME_SOURCE="<source_account_name>"
STORAGE_KEY_SOURCE="<source_account_key>"
STORAGE_NAME_DEST="<dest_account_name>"
STORAGE_KEY_DEST="<dest_account_key"

CONNECTION_STRING_SOURCE="DefaultEndpointsProtocol=https;AccountName=$STORAGE_NAME_SOURCE;AccountKey=$STORAGE_KEY_SOURCE"
CONNECTION_STRING_DEST="DefaultEndpointsProtocol=https;AccountName=$STORAGE_NAME_DEST;AccountKey=$STORAGE_KEY_DEST"

containers=$(az storage container list --connection-string $CONNECTION_STRING_SOURCE --output tsv | awk '{print $2}')
for i in $containers;
do
    docker run --rm incendonet/azcopy azcopy --source https://$STORAGE_NAME_SOURCE.blob.core.windows.net/$i --source-key $STORAGE_KEY_SOURCE --destination https://$STORAGE_NAME_DEST.blob.core.windows.net/$i --dest-key $STORAGE_KEY_DEST --recursive
    access_level=$(az storage container show-permission -n $i --connection-string $CONNECTION_STRING_SOURCE --output tsv)
    az storage container set-permission -n $i --connection-string $CONNECTION_STRING_DEST --public-access $access_level    
done

MongoDB select fields after $lookup

When there is a $lookup stage to join a list of large documents, an error Total size of documents in ... matching ... exceeds maximum document size can arrive.

It’s possible to avoid this with $unwind stage right after $lookup. More explanations in the documentation. And then the documents can be regrouped with the required fields.

Order.objects.aggregate(
    {
        '$lookup': {
            'from': 'item',
            'localField': '_id',
            'foreignField': 'order_id',
            'as': 'items'
        }
    },
    {
        "$unwind": "$items"
    },
    {
        "$group": {
            "_id": "$_id",
            "date": {"$last": "$date"},
            "items": {
                "$push": {
                    "name": "$items.name",
                    "price": "$items.price"
                }
            }
        }
    }
)

How to find a change for a field with MongoDB aggregation

For example there is a collection device_status which stores the different states for the devices. The task is to find the devices which passed from off to on at least one time.

{ "device" : "device1", "state" : "on", "ts": ISODate("2018-06-07T17:05:29.340+0000") }
{ "device" : "device2", "state" : "off", "ts": ISODate("2018-06-08T17:05:29.340+0000") }
{ "device" : "device3", "state" : "on", "ts": ISODate("2018-06-09T17:05:29.340+0000")}
{ "device" : "device3", "state" : "shutdown", "ts": ISODate("2018-06-09T18:05:29.340+0000")}
{ "device" : "device2", "state" : "load", "ts": ISODate("2018-06-09T19:05:29.340+0000") }
{ "device" : "device2", "state" : "on", "ts": ISODate("2018-06-10T17:05:29.340+0000") }
{ "device" : "device3", "state" : "off", "ts": ISODate("2018-06-11T17:05:29.340+0000") }
{ "device" : "device1", "state" : "idle", "ts": ISODate("2018-06-11T18:05:29.340+0000") }
{ "device" : "device3", "state" : "on", "ts": ISODate("2018-06-12T17:05:29.340+0000") }
...

The first stage is to sort the data by device and date.

1/8 »