python

SINGLETON factory

You can define Singleton class manually, but if you want to create multiple different Singleton classes on the fly then you can use the function below :

#Factory : to create Singleton classes
def singleton(new_klass, klass): 

	def new(cls, *args, **kvargs):
		if not hasattr(cls, 'instance'):
			cls.instance = klass(*args, **kvargs)
		return cls.instance

	globals()[new_klass] = type(new_klass, (object,), {'__new__' : new})

Progress bar

def progress(percent=0, width=30):
    left = width * percent // 100
    right = width - left
    print('\r[', '#' * left, ' ' * right, ']',
          f' {percent:.0f}%',
          sep='', end='', flush=True)

Bi-directional dictionary

import os
import json

class BiDict(dict):
	def __init__(self, *args, **kwargs):
		super().__init__(*args, **kwargs)
		self.inverse = {}
		for key, value in self.items():
			self.inverse.setdefault(value,key) 

	def __setitem__(self, key, value):
		if key in self : del self.inverse[self[key]]
		super().__setitem__(key, value)
		self.inverse.setdefault(value,key)		

	def __delitem__(self, key):
		value = self[key]
		if value in self.inverse : del self.inverse[value]
		super().__delitem__(key)

	def pop(self,key):
		value = super().pop(key)
		self.inverse.pop(value)
		return value

	def load(self, path, name):
		assert os.path.isdir(path), f'dir: {path} does not exists'

		full = f'{path}/{name}'
		with open(full,'r') as fh : self.__init__(json.load(fh))


	def save(self, path, name):
		assert os.path.isdir(path), f'dir: {path} does not exists'

		full = f'{path}/{name}'
		with open(full,'w') as fh :
			json.dump(self, fh, sort_keys=True, indent=2)

class AttrBiDict(BiDict):
	def __init__(self, *args, **kwargs):
		super(AttrBiDict, self).__init__(*args, **kwargs)
		self.__dict__ = self

	def __hash__(self): return hash(str(self))
	def __eq__(self,other): return self.__hash__() == other.__hash__()

Timing code

This one is quick.

Java

Here is how you do it :

import java.lang.Thread;  

public class Main {
  public static void main(String[] args)  throws Exception {
        
    long startTime = System.nanoTime();
    Thread.sleep(3000);
    long endTime = System.nanoTime();
    //divide by 1000000 to get milliseconds.
    int duration = (int) ((endTime - startTime) / (1000000 * 1000));  

    System.out.println("it took: " + duration + " secs");
  }
}

-----

it took: 3 secs

Prolog

?- numlist(1, 100, L).
L = [1, 2, 3, 4, 5, 6, 7, 8, 9|...].

?- time(numlist(1, 100, L)).
% 105 inferences, 0.000 CPU in 0.000 seconds (94% CPU, 1126344 Lips)
L = [1, 2, 3, 4, 5, 6, 7, 8, 9|...].

Python

For Python look at this post : Timing code execution

Autovivify in Python

Autovivify is simply fancy word for the behavior of creating a dictionary key-value automatically the moment an element is accessed. Python throws an error if the kv-pair was not already created.

Perl hashes/dictionaries are auto-vivified by default, not so for Python, which could be annoying, because you have to do additional checks thus breaking the normal flow of coding.

Here is how you make a auto vivifying Python dictionary and at the same time quick way to create a Tree structure :

from collections import defaultdict
from pprint import pprint

def Tree():
    return defaultdict(Tree)

tree = Tree()
tree['mammal']['ape']['chimp'] = 'bobo'

pprint(tree,indent=2,compact=True)

-----

defaultdict(...,
  { 'mammal': defaultdict(...,
    { 'ape': defaultdict(...,
      { 'chimp': 'bobo'})})})

Even better, if you want cleaner print :

from collections import defaultdict
from pprint import pprint

def Tree(): return DD(Tree)

class DD(defaultdict):
    __repr__ = dict.__repr__

tree = Tree()
tree['mammal']['ape']['chimp'] = 'bobo'

pprint(tree,indent=2,compact=True)

------

{'mammal': {'ape': {'chimp': 'bobo'}}}

Numpy: Filtering by multiple conditions

Apart from using np.where() a different way to filter a numpy array is to use the following syntax :

ary[ condition ]

for example :

ary[ ary > 10 ]

So far so good, but how do you apply multiple conditions. To do this you need to surround all the conditions in brackets as shown below. Also for logical operators you should use the bitwise operators instead.

import numpy as np

a = np.arange(5,15)

print(a)

#one condition
print(a[a > 10])
#multiple conditions
print(a[(a > 10) & (a < 13)])
print(a[(a == 10) | (a == 13)])

-----

[ 5  6  7  8  9 10 11 12 13 14]
[11 12 13 14]
[11 12]
[10 13]

Call method by String or Reference

Sometimes you may need to call a method, but you don’t know which one in advance i.e. you want to postpone the decision to runtime.
There are two basic ways to that :

  • Store the method name as a String and build the “call” on the fly when you need it
  • Store a reference to a method in a variable and then use language specific syntax to invoke the method

For example let say you have written multiple standalone tests and you want to run different combination of them based on different needs. In other words you want to parameterize method calls.

Something along the lines :

run_tests -suite=3,5,8
run_tests -suite=2..5

If you have a few combinations you can hard code them, but it is more flexible to “pick” the tests/methods on the fly.

This is just one example of the need for selecting the function/method at the last moment. In most cases you prepare or store the name in a String.

What follows are examples of how to do this in several languages.

Python

There are many different ways to call a function or a method indirectly. Below you can see most of them.

#shortcut of : def say(*a): print(*a)
say = print

#using eval()
eval("say")("hello")
say_hi = "say('hi')"
eval(say_hi)

#call defined functon via locals
locals()["say"]('boo hoo')

#via reference stored in a dict
funs = { 'sayit' : say }
funs['sayit']('sayit')

# ----- and now ... -----

#methods in a string
class StringMethod:
  
  def say(self, *args) : print(*args)
  
  
sm = StringMethod()

#"create" a method
method = getattr(sm,"say")
method("howdy")

#direct assignment
sm_say = sm.say
sm_say('whats sup')

#unbound
sayit = StringMethod.say
sayit(sm,'bye')


-----

hello
hi
boo hoo
sayit
howdy
whats sup
bye

Java

In Java you have to use Reflection. You do it in two steps :

  • First you create a Method out of the String and a description of the parameters
  • Then you call the .invoke() method with the real values

import java.util.*;
import java.lang.reflect.*;


public class MethodReflection {

  public static <ARG> void say(ARG arg) { System.out.println(arg); }

  public void say_hi() { say("hi"); }
  private void sayit(String str) { say(str); }
  
  public void test() throws Exception {
    //get the class that hold the methods you want to call
    Class<?> klass = MethodReflection.class;//Class.forName("MethodReflection");
    
    //get a method w/o arguments
    Method m1 = klass.getMethod("say_hi");// or getDeclaredMethod()
    m1.invoke(this);
    
    //private method requires getDeclaredMethod(), also using one String argument
    Method m2 = klass.getDeclaredMethod("sayit", String.class);
    m2.invoke(this, "hello");
    
    //unknown argument
    Method m3 = klass.getMethod("say", Object.class);
    //first argument null for static method
    m3.invoke(null, "howdy");
    
  }

  public static void main(String[] args) throws Exception {
    MethodReflection mc = new MethodReflection(); 
    mc.test();
  }
}

-----

hi
hello
howdy

JavaScript

say = console.log

eval("say('hi')")

sayit = "say"
window[sayit]('sayit')

-----

hi
sayit

Slices, Ranges and __getitem__()

One annoyance when implementing __getitem__() is debugging the slice syntax interactively .. Below you can see an example.

First you create a dummy class with simple method that print its arguments.

Then you can experiment.

class Blah: 
  def __getitem__(self,*args): 
    print(args) 
    return args

b = Blah()
b[1]
b[2:5]
b[::2,2:11:2]

s = b[::3]

print()

#filling the indices
ind = s[0].indices(12)
print(f'indices: {s[0]} => {ind}')

#make it a range
rng = range(*s[0].indices(12))
print(f'range: {rng}')

for i in rng: print(i)
(1,)
(slice(2, 5, None),)
((slice(None, None, 2), slice(2, 11, 2)),)
(slice(None, None, 3),)

indecies: slice(None, None, 3) => (0, 12, 3)
range: range(0, 12, 3)
0
3
6
9

In addition inside your method you can convert a slice to a range by using the .indices() method … and then use them to do processing in a loop.

Incremental average

There are three ways to calculate an Average depending on the way we receive the data :

  1. Basic : we have all the data. In this case we just use the basic well known formula : $$Avg = \frac{1}{n} \sum_{i=0}^n x_i$$
  2. Moving average : calculated using rolling window
  3. Incremental average : the one that we will discuss now

The idea is to calculate the Basic average at every step w/o recalculating it from the whole sequence i.e. the data comes one value at a every time step.

Here is the formula :

$$a_n = a_{n-1} + \frac{x_n – a_{n-1}}{n}$$

here is how you can use it as a python closure function, so that you don’t have to carry the state :

def iavg():
  avg = 0
  n = 0
  def calc(value):
    nonlocal n,avg
    n += 1
    avg = avg + ((value - avg) / n)
    return avg
  return calc
  
avg = iavg()
print(f'2 => {avg(2)}')
print(f'4 => {avg(4)}')
print(f'6 => {avg(6)}')

-----

2 => 2.0
4 => 3.0
6 => 4.0

# (2+4+6) / 3 = 12/3 = 4

Timing code execution

Python have timeit module to test how long a piece of code takes to execute. On the other ipython %timeit magic gives much more useful information.

$ python3 -m timeit -s "5 == 55"
100000000 loops, best of 3: 0.00543 usec per loop

In [23]: %timeit 5 == 55                                                                                                                                                     
18.3 ns ± 0.393 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)

Luckily @Numerlor was kind enough to extract the functionality from ipython for us

In [26]: nice_timeit('5 == 55')                                                                                                                                              
18.4 ns +- 0.328 ns per loop (mean +- std. dev. of 7 runs, 10000000 loops each)
import math
import timeit


def _format_time(timespan, precision=3):
    """Formats the timespan in a human readable form"""
    units = ["s", "ms", "\xb5s", "ns"]
    scaling = [1, 1e3, 1e6, 1e9]
    if timespan > 0.0:
        order = min(-int(math.floor(math.log10(timespan)) // 3), 3)
    else:
        order = 3
    scaled_time = timespan * scaling[order]
    unit = units[order]
    return f"{scaled_time:.{precision}g} {unit}"


class TimeitResult(object):
    """
    Object returned by the timeit magic with info about the run.

    Contains the following attributes :

    loops: (int) number of loops done per measurement
    repeat: (int) number of times the measurement has been repeated
    best: (float) best execution time / number
    all_runs: (list of float) execution time of each run (in s)
    compile_time: (float) time of statement compilation (s)
    """

    def __init__(self, loops, repeat, best, worst, all_runs, compile_time, precision):
        self.loops = loops
        self.repeat = repeat
        self.best = best
        self.worst = worst
        self.all_runs = all_runs
        self.compile_time = compile_time
        self._precision = precision
        self.timings = [dt / self.loops for dt in all_runs]

    @property
    def average(self):
        return math.fsum(self.timings) / len(self.timings)

    @property
    def stdev(self):
        mean = self.average
        return (
            math.fsum([(x - mean) ** 2 for x in self.timings]) / len(self.timings)
        ) ** 0.5

    def __str__(self):
        return "{mean} {pm} {std} per loop (mean {pm} std. dev. of {runs} run{run_plural}, {loops} loop{loop_plural} each)".format(
            pm="+-",
            runs=self.repeat,
            loops=self.loops,
            loop_plural="" if self.loops == 1 else "s",
            run_plural="" if self.repeat == 1 else "s",
            mean=_format_time(self.average, self._precision),
            std=_format_time(self.stdev, self._precision),
        )


def nice_timeit(
    stmt="pass",
    setup="pass",
    number=0,
    repeat=None,
    precision=3,
    timer_func=timeit.default_timer,
    globals=None,
):
    """Time execution of a Python statement or expression."""

    if repeat is None:
        repeat = 7 if timeit.default_repeat < 7 else timeit.default_repeat

    timer = timeit.Timer(stmt, setup, timer=timer_func, globals=globals)

    # Get compile time
    compile_time_start = timer_func()
    compile(timer.src, "<timeit>", "exec")
    total_compile_time = timer_func() - compile_time_start

    # This is used to check if there is a huge difference between the
    # best and worst timings.
    # Issue: https://github.com/ipython/ipython/issues/6471
    if number == 0:
        # determine number so that 0.2 <= total time < 2.0
        for index in range(0, 10):
            number = 10 ** index
            time_number = timer.timeit(number)
            if time_number >= 0.2:
                break

    all_runs = timer.repeat(repeat, number)
    best = min(all_runs) / number
    worst = max(all_runs) / number
    timeit_result = TimeitResult(
        number, repeat, best, worst, all_runs, total_compile_time, precision
    )

    # Check best timing is greater than zero to avoid a
    # ZeroDivisionError.
    # In cases where the slowest timing is lesser than a microsecond
    # we assume that it does not really matter if the fastest
    # timing is 4 times faster than the slowest timing or not.
    if worst > 4 * best and best > 0 and worst > 1e-6:
        print(
            f"The slowest run took {worst / best:.2f} times longer than the "
            f"fastest. This could mean that an intermediate result "
            f"is being cached."
        )

    print(timeit_result)

    if total_compile_time > 0.1:
        print(f"Compiler time: {total_compile_time:.2f} s")
    return timeit_result


# nice_timeit("time.sleep(0.3)", "import time")

# IPython license
# BSD 3-Clause License
#
# - Copyright (c) 2008-Present, IPython Development Team
# - Copyright (c) 2001-2007, Fernando Perez <fernando.perez@colorado.edu>
# - Copyright (c) 2001, Janko Hauser <jhauser@zscout.de>
# - Copyright (c) 2001, Nathaniel Gray <n8gray@caltech.edu>
#
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice, this
#   list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above copyright notice,
#   this list of conditions and the following disclaimer in the documentation
#   and/or other materials provided with the distribution.
#
# * Neither the name of the copyright holder nor the names of its
#   contributors may be used to endorse or promote products derived from
#   this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.