Watching for changes on a list of file

entr is a Linux command to watch a list of files

# Watch a list of python files and restart server on changes
find ./ -name "*.py" | entr python app.py

• Caveat : this does not react on new files.

Watching for changes and new files in existing dirs

# Watch a dir and restart server on changes and new files in existing dirs
while true
do 
    find . -name "*.py" | entr -d python ./app.py 
done

• The -d option stops makes entr exit on new files, so the list of files to watch is recomputed.
• Caveat : this does not watch new directories !

Definition

A generic function is a function defined for polymorphism.

Implementation

In Python3.6, you can achieve this using functools.singledispatch:

from functools import singledispatch

@singledispatch
def myfunc(arg, verbose=False):
  if verbose:
    print('Length of strings')
  print('This is not a string')
 
@myfunc.register(str)
def _(arg, verbose=False):
  print(len(arg))
  
 
print(myfunc(3))
# >> 'This is not a string'

print(myfunc('abc'))
# >> 3

To spy the stdout and stderr of a process, run:

sudo strace -s9999 -e write -p <PID>

This can be useful when a script is not run by you and you want to get the stderr.

Source: Monsanto

Add/change a key to a JSON

function jq_add_key {
  jq --arg key "$2" ". + {\"$1\": \$key}"
}

cat <source>.json | jq --arg key <value> '. + {<key>: $key}'

cat {source}.json | jq_change_key {key} {new_value} > {dest}.json

Modify a key only if exists in a JSON

function jq_change_key {
    jq "to_entries |
        map(if .key == \"$1\"
            then . + {\"value\":\"$2\"}
            else .
            end) |
        from_entries"
}                                      

cat {source}.json | jq_change_key {key} {new_value} > {dest}.json

Caution: do not replace {source}.json as you cat it, it would result in a blank file !

Source: Lob’s blog

Some time, when the tables get bigger, the request time of some requests can be incredibly high (5 minutes for a join on a table). The explaination can come from the tables not being properly vacuumed or analyzed.

Vacuuming cleans up stale or temporary data, and analyzing refreshes its knowledge of all the tables for the query planner. We saw an immediate decrease in execution time for our complex queries, and as a result, a much more user-friendly internal website.

Diagnostic

To see the last time the tables has been vaccumed and analyzed

SELECT relname, last_vacuum, last_autovacuum, last_analyze, last_autoanalyze
FROM pg_stat_all_tables WHERE schemaname = 'public';

Single shot vacuum

VACUUM ANALYZE table_name;

Set-up automation

ALTER TABLE table_name SET (autovacuum_vacuum_scale_factor = 0.0);
ALTER TABLE table_name SET (autovacuum_vacuum_threshold = 5000);

Tables are auto-vacuumed when the number of row inserted is greater than the corresponding threshold

vacuum threshold = autovacuum_vacuum_threshold + autovacuum_vacuum_scale_factor * number of rows in table