myriad

Split a List in pieces

How would you SPLIT a List in pieces of specified length ? …. Here is how :

:- initialization(main).

say(L) :- write(L), write('\n').

%%this illustrates how to pick part of a List
firstN(Lst,N,FirstN) :- length(FirstN,N), append(FirstN,_Rest,Lst).

split([],_,[]) :- !.
split(Lst, N, [FirstN|Res]) :- 
   length(FirstN, N), append(FirstN, Rest, Lst), !, split(Rest, N, Res).
%%when the last piece is smaller than the requested size
split(Lst, N, [Lst]) :- length(Lst, Len), Len < N.

main :- 
L = [1,2,3,4,5,6], say(L),
firstN(L,5,F), say(F),
split(L,2,L2), say(L2),
split(L,3,L3), say(L3),
split(L,4,L4), say(L4).

----

[1,2,3,4,5,6]
[1,2,3,4,5]
[[1,2],[3,4],[5,6]]
[[1,2,3],[4,5,6]]
[[1,2,3,4],[5,6]]

Arrays in Prolog

Prolog variables are immutable. for this reason there is no builtin array type.

For Arrays implemented using Prolog OOP check here

Think about it to support backtracking Prolog have keep the state at every point. First arrays can be big and vector operations can involve elements all over the place. So keeping track is not just inefficient, but also very complex.

For this and other reasons Prolog does not support standard type of arrays.

That is all good and stuff, but what if you still need simple small array !?
The obvious way to implement it is to use Lists, but because the access to elements is sequential the implementation will be slow.

Instead we will do it in a more hacky, no-no method, by using Terms instead of Lists. We do that by employing arg and setarg. The difference is that they allow direct access ergo implementation will be fast.

Below you have simple API that support 1D and 2D arrays.

:- initialization(main).

say(X) :- write(X), write('\n').
say(D,X) :- write(D), write(X), write('\n').

ary1d_new(Size,Sym,Ary) :- 
	functor(Ary,a1,Size), 
	forall(arg(X,Ary,_), nb_setarg(X,Ary,Sym)).
ary1d_get(Pos,Ary,Val) :- arg(Pos,Ary,Val).
ary1d_set(Pos,Ary,Val) :- nb_setarg(Pos,Ary,Val).

ary2d_new(Rows,Cols,Sym,Ary) :- 
	functor(Ary,a2,Rows), 
	ary1d_new(Cols,Sym,Row),
	forall(arg(X,Ary,_), nb_setarg(X,Ary,Row)).
ary2d_get(X,Y,Ary,Val) :- arg(X,Ary,Row), arg(Y,Row,Val).
ary2d_set(X,Y,Ary,Val) :- arg(X,Ary,Row), nb_setarg(Y,Row,Val).

main :-
say('1D example : '),
ary1d_new(5,0,A1D),say('new(5) : ', A1D),
ary1d_set(2,A1D,1),say('set(2,1) : ', A1D),
ary1d_get(2,A1D,Val),say('get(2) : ', Val),
say('2D example : '),
ary2d_new(3,3,0,A2D),say('new(3,3) : ', A2D),
ary2d_set(2,2,A2D,1),say('set(2,2,1) : ', A2D),
ary2d_get(2,2,A2D,Val),say('get(2,2) : ', Val),
ary2d_get(X,Y,A2D,1),say('find coord of 1 : ', [X,Y]).

-----

1D example : 
new(5) : a1(0,0,0,0,0)
set(2,1) : a1(0,1,0,0,0)
get(2) : 1
2D example : 
new(3,3) : a2(a1(0,0,0),a1(0,0,0),a1(0,0,0))
set(2,2,1) : a2(a1(0,0,0),a1(0,1,0),a1(0,0,0))
get(2,2) : 1
find coord of 1 : [2,2]
true.

The implementation of 2D array above uses as storage mechanism 1D array for every row.

But there is also possible to use single 1D array for this purpose. In this case to access a cell we would need to transform the 2D coordinate to 1D.

One problem with this is there is no way to guess the dimensions of the 2D array if we use 1D array to represent it. For this reason we will use the first 2 elements of the storage to store those dimensions.

Here is how an implementation will look like :

%% 2D array using 1D storage
a2d_new(Rows,Cols,Sym,Ary) :- 
	Size is Rows * Cols + 2, 
	ary1d_new(Size,Sym,Ary), 
	nb_setarg(1,Ary,Rows), nb_setarg(2,Ary,Cols).
a2d_get(X,Y,Ary,Val) :- 
	arg(2,Ary,Cols), Pos is (X-1) * Cols + Y + 2,
	arg(Pos,Ary,Val).
a2d_set(X,Y,Ary,Val) :- 
	arg(2,Ary,Cols), Pos is (X-1) * Cols + Y + 2,
	nb_setarg(Pos,Ary,Val).
a2d2lst(Ary,Lst) :- Ary =.. [ _,_,_ | Lst].

For Arrays implemented using Prolog OOP check here

Autovivify in Python

Autovivify is simply fancy word for the behavior of creating a dictionary key-value automatically the moment an element is accessed. Python throws an error if the kv-pair was not already created.

Perl hashes/dictionaries are auto-vivified by default, not so for Python, which could be annoying, because you have to do additional checks thus breaking the normal flow of coding.

Here is how you make a auto vivifying Python dictionary and at the same time quick way to create a Tree structure :

from collections import defaultdict
from pprint import pprint

def Tree():
    return defaultdict(Tree)

tree = Tree()
tree['mammal']['ape']['chimp'] = 'bobo'

pprint(tree,indent=2,compact=True)

-----

defaultdict(...,
  { 'mammal': defaultdict(...,
    { 'ape': defaultdict(...,
      { 'chimp': 'bobo'})})})

Even better, if you want cleaner print :

from collections import defaultdict
from pprint import pprint

def Tree(): return DD(Tree)

class DD(defaultdict):
    __repr__ = dict.__repr__

tree = Tree()
tree['mammal']['ape']['chimp'] = 'bobo'

pprint(tree,indent=2,compact=True)

------

{'mammal': {'ape': {'chimp': 'bobo'}}}

Numpy: Filtering by multiple conditions

Apart from using np.where() a different way to filter a numpy array is to use the following syntax :

ary[ condition ]

for example :

ary[ ary > 10 ]

So far so good, but how do you apply multiple conditions. To do this you need to surround all the conditions in brackets as shown below. Also for logical operators you should use the bitwise operators instead.

import numpy as np

a = np.arange(5,15)

print(a)

#one condition
print(a[a > 10])
#multiple conditions
print(a[(a > 10) & (a < 13)])
print(a[(a == 10) | (a == 13)])

-----

[ 5  6  7  8  9 10 11 12 13 14]
[11 12 13 14]
[11 12]
[10 13]

Call method by String or Reference

Sometimes you may need to call a method, but you don’t know which one in advance i.e. you want to postpone the decision to runtime.
There are two basic ways to that :

  • Store the method name as a String and build the “call” on the fly when you need it
  • Store a reference to a method in a variable and then use language specific syntax to invoke the method

For example let say you have written multiple standalone tests and you want to run different combination of them based on different needs. In other words you want to parameterize method calls.

Something along the lines :

run_tests -suite=3,5,8
run_tests -suite=2..5

If you have a few combinations you can hard code them, but it is more flexible to “pick” the tests/methods on the fly.

This is just one example of the need for selecting the function/method at the last moment. In most cases you prepare or store the name in a String.

What follows are examples of how to do this in several languages.

Python

There are many different ways to call a function or a method indirectly. Below you can see most of them.

#shortcut of : def say(*a): print(*a)
say = print

#using eval()
eval("say")("hello")
say_hi = "say('hi')"
eval(say_hi)

#call defined functon via locals
locals()["say"]('boo hoo')

#via reference stored in a dict
funs = { 'sayit' : say }
funs['sayit']('sayit')

# ----- and now ... -----

#methods in a string
class StringMethod:
  
  def say(self, *args) : print(*args)
  
  
sm = StringMethod()

#"create" a method
method = getattr(sm,"say")
method("howdy")

#direct assignment
sm_say = sm.say
sm_say('whats sup')

#unbound
sayit = StringMethod.say
sayit(sm,'bye')


-----

hello
hi
boo hoo
sayit
howdy
whats sup
bye

Java

In Java you have to use Reflection. You do it in two steps :

  • First you create a Method out of the String and a description of the parameters
  • Then you call the .invoke() method with the real values

import java.util.*;
import java.lang.reflect.*;


public class MethodReflection {

  public static <ARG> void say(ARG arg) { System.out.println(arg); }

  public void say_hi() { say("hi"); }
  private void sayit(String str) { say(str); }
  
  public void test() throws Exception {
    //get the class that hold the methods you want to call
    Class<?> klass = MethodReflection.class;//Class.forName("MethodReflection");
    
    //get a method w/o arguments
    Method m1 = klass.getMethod("say_hi");// or getDeclaredMethod()
    m1.invoke(this);
    
    //private method requires getDeclaredMethod(), also using one String argument
    Method m2 = klass.getDeclaredMethod("sayit", String.class);
    m2.invoke(this, "hello");
    
    //unknown argument
    Method m3 = klass.getMethod("say", Object.class);
    //first argument null for static method
    m3.invoke(null, "howdy");
    
  }

  public static void main(String[] args) throws Exception {
    MethodReflection mc = new MethodReflection(); 
    mc.test();
  }
}

-----

hi
hello
howdy

JavaScript

say = console.log

eval("say('hi')")

sayit = "say"
window[sayit]('sayit')

-----

hi
sayit

Switching code to a mirrored repo

Let say you mirror your repository from gitlab to github or vs versa. You can re-clone your code from the new place.

But there is a better and faster way. First it is good idea to view your current config …

git remote -v

Then to do the change do something like this :

git remote set-url origin git@new_repo.com/something/blah.git

Now check if it changed :

git remote -v

Padding values …

Here is an example of how to pad values with fill characters :

public class Main {
  
  public static <T> String pad(T str, int n, String right_pad, String filler) {
    return String.format("%1$" + right_pad + n + "s", str).replaceAll(" ", filler);
  }
  
  public static <T> String pad_left(T str, int n, String filler) {
    return pad(str, n, "", filler);
  }  
  public static <T> String pad_right(T str, int n, String filler) {
    return pad(str, n, "-", filler);
  }  
  
  public static <T> String pad_left(T str, int n) {
    return pad(str, n, "", " ");
  }  
  public static <T> String pad_right(T str, int n) {
    return pad(str, n, "-", " ");
  }  
  
  public static void main(String[] args) {
      System.out.println("pad(5,3)     : " + pad_left(5,3));
      System.out.println("pad(5,3,\"0\") : " + pad_left(5,3,"0"));
      System.out.println("pad(5,3,\"-\") : " + pad_right(5,3,"-"));
  }
}

-------

pad(5,3)     :   5
pad(5,3,"0") : 005
pad(5,3,"-") : 5--

Calculating date difference

Here is a quick way to calculate the difference between two dates counted in days.
You can easily modify it to use different TimeUnit.

import java.util.*;
import java.util.concurrent.TimeUnit;

public class Main {
  
  public static int days_diff(Date from, Date to) {
    long time_diff = Math.abs(to.getTime() - from.getTime() );
    int days = (int) TimeUnit.DAYS.convert(time_diff, TimeUnit.MILLISECONDS);
    return days;
  }

  public static void main(String[] args) {
      Date now = new Date();
      //use year - 1900, and month counting starts from Zero
      Date past = new Date(2022 - 1900, 3 - 1,1);
      
      System.out.println("now: " + past);
      System.out.println("now: " + now);
      System.out.println("Days difference : " + days_diff(past,now));
  }
}

-----------------------

now: Tue Mar 01 00:00:00 UTC 2022
now: Wed Mar 23 14:24:54 UTC 2022
Days difference : 22

Managing nested structures in Java

Handling nested structures in Java is kind of a nightmare. For this reason I created EasyLoL.

LoL is abbreviation of List-of-List, a general way of calling nested structures.

Imagine having to build structure like that in pure Java, how many keystroke energy you need to waste. No more just use EasyLoL and you will have the last laugh …

a : >
  b : >
    c : ccc
    d : ddd
  f : >
    c : fff2
    e : [abra,cadabra]
bcd : bcd
abc : abc
structs : >
  list : [v1,v2,v3]
  hash : >
    k1 : v1
    k2 : v2
    k3 : v3

The module extends HashMap and builds on top of it.

It provides fetch(), set() and delete() methods that use a dotted syntax to access and instantiate an element anywhere in the hierarchy by autovivifing all the intermediary elements if necessary.

autovivify : is the automatic creation of new arrays and hashes as required every time an undefined value is dereferenced.

The module also support a quick way to create and insert on the fly a HashMap or ArrayList element, you do that by encoding it as string, pre-pending it with special character, so that the module can figure out which one do you want to create.
Here is an example :

//hash in a string
e.set("structs.hash", ">k1:v1,k2:v2,k3:v3");
// list in a string
e.set("structs.list", "]v1,v2,v3");	

Below you can see the class and examples of how to use it.
Also you can also use dump() function I described in a previous post to pretty print the data structure.

import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

// Easy way to Create and Access hierarchical structures
public class EasyLoL extends HashMap<String, Object> {
	
		public static <ARG> void say(ARG arg) { System.out.println(arg); }

		//extract value by going trough the hierarchy
		private Object _by_key(String[] keys, HashMap val) {
			Object rv = null;
			if ( val.containsKey(keys[0])) { rv = val.get(keys[0]); }
			else {
				//log.debug("! by_key> no such key : " + keys[0]);
				return null; 
			}
			// dig deeper, (skim the first element of String keys)
			if (rv instanceof HashMap && keys.length > 1 ) return _by_key(Arrays.copyOfRange(keys, 1, keys.length), (HashMap) rv);
			if (keys.length > 1) return null; //still more keys, but no more data that deep 
			return rv;
		}
		
		//helper method for LoL objects access
		public Object by_key(String key) {
			String[] keys = key.split("\\.");
			return _by_key(keys, this);
		}
		
		//Getter methods ....
		public Object find(String key) { return key.contains(".") ? by_key(key) : get(key); }
			
		public String gets(String key) {
			Object obj = find(key);
			if (obj instanceof String) return (String) obj;
			if (obj == null) return null;
			return obj.toString();
	 	}
		
		public void delete(String key) {
			int i = key.lastIndexOf('.') ;
			if (i == -1) return;
			String upto = key.substring(0,i);
			String last = key.substring(i+1);
			EasyLoL prev = fetch(upto);
			prev.remove(last);
		}
		
		public <T> T fetch(String key) {
			Object obj = find(key);
			if (obj == null) return null;
			return (T) obj;
	 	}
		
		public ArrayList by_keys(String ... keys) {
			ArrayList<String> lst = new ArrayList<>();
			for (String key : keys) lst.add(gets(key));
			return lst;
		}
		
		//converts string to Hash
		public HashMap<String,String> str2lol(String str) {
			HashMap<String,String> hash = new HashMap<String,String>();
			String[] kvs = str.split("[,:]");
			for (int i=0; i < kvs.length; i += 2) hash.put(kvs[i], kvs[i+1]);
			return hash;
		}
		
		//handles setting complex types
		private <T> void insert(EasyLoL lol, String k, T value) {
			//hash encoded in string
			if (value instanceof String) {
				if (((String) value).startsWith(">")) {
					HashMap<String,String> val = str2lol( ((String) value).substring(1) );
					lol.put(k, val);
					return;
				} else if (((String) value).startsWith("]")) {
					String val = ((String) value).substring(1);
					lol.put(k, Arrays.asList(val.split(",")));
					return;
				}
			} 
			lol.put(k, value);			
		}
		
		public <T> void set(String key, T value) throws Exception { this.set(key, value, true); }
		
		public <T> void set(String key, T value, boolean silent) throws Exception {
			
			//no hierarchy
//			if (! (key.contains(".") && this.containsKey(key))) {
//				this.put(key, value);
//			}
			
			String[] keys = key.split("\\.");
			//start point
			EasyLoL lol = this;
			int pos = 0;//are we at a leaf 
			//incrementally vivify the keys
			for (String k : keys) {
				if (lol.containsKey(k)) {
					if (lol.get(k) instanceof Map) {
						lol = (EasyLoL) lol.get(k);//extend
					} else {
						if (pos >= keys.length-1) {
							insert(LoL,k,value);//overwrite
							return;
						}	
						if (silent) return;
						throw new Exception("using : " + key + " ! Can't overwrite the structure at sub-key : " + k);
					}
				} else {//autovivify
					if (pos < keys.length-1) {//Intermediary keys
						LoL.put(k, new EasyLoL());
						//switch/move one level down
						LoL = (EasyLoL) LoL.get(k);
					} else {//tree leaf, set the value
						insert(LoL,k,value);
					}
				}
				pos++;
			}
	 	}
		
		
	public static void main(String[] args) throws Exception {
		EasyLoL e = new EasyLoL();
		e.set("abc", "abc");
		//Hierarchical set
		e.set("a.b.c", "ccc");
		e.set("a.b.d", "ddd");

		e.set("a.f.c", "fff");
		//overwrite
		e.set("a.f.c", "fff2");
		//conflict : structure already in place
		e.set("a.f.c.x", "fffx");//silent
		//e.set("a.f.c.x", "fffx",false);//throw error
		
		//setting java structure
		ArrayList ary = new ArrayList() {{
			add("abra");
			add("cadabra");
		}};
		e.set("a.f.e", ary);
		
		e.set("bcd", "bcd");
		//hash in a string
		e.set("structs.hash", ">k1:v1,k2:v2,k3:v3");
		// list in a string
		e.set("structs.list", "]v1,v2,v3");
		
		say("------------------------------------------");
		say("a.b : " + e.fetch("a.b"));
		say("a.b.c : " + e.fetch("a.b.c"));
		say("structs.list : " + e.fetch("structs.list"));
		say("structs.hash : " + e.fetch("structs.hash"));
//		e.delete("a.b");
		say("==========================================");
//		say(utils.dump(e));
	}

}

------------------------------------------
a.b : {c=ccc, d=ddd}
a.b.c : ccc
structs.list : [v1, v2, v3]
structs.hash : {k1=v1, k2=v2, k3=v3}
==========================================
a : >
  b : >
    c : ccc
    d : ddd
  f : >
    c : fff2
    e : [abra,cadabra]
bcd : bcd
abc : abc
structs : >
  list : [v1,v2,v3]
  hash : >
    k1 : v1
    k2 : v2
    k3 : v3