Company Contractor Cash Flow Diagram

code:

digraph G {

subgraph cluster_0 {
label = “Me”;
node [style=filled];
pension salary dividend mbalance ISA stocks expenses;
color=blue
}

subgraph cluster_1 {
node [style=filled];
NIC VAT CapitalGains IncomeTax Corporation;
label = “Gov”;
color=red
}

subgraph cluster_2 {
node [style=filled];
profits revenue cbalance employee;
label = “Company”;
}

mbalance -> pension;
IncomeTax -> pension [label=”tax relief”];
mbalance -> stocks;
stocks -> mbalance;
stocks -> CapitalGains [label=”10% tax”];
ISA -> mbalance;
mbalance -> costs;
revenue -> pension [label=”40K limit”];
revenue -> VAT [label=”14.5% VAT on revenue”];
cbalance -> dividend [label=”pay shareholders”];
“UK customer” -> revenue [label=”pays for services\n+20% VAT”];
customer -> revenue;
revenue -> profits;
profits -> cbalance;
revenue -> costs;
cbalance -> employee;
profits -> Corporation [label=”19% of profits”];
employee -> NIC;
employee -> salary;
salary -> IncomeTax [label=”20-45% tax”];
costs -> expenses [label=””];
pension -> mbalance;
dividend -> mbalance [label=”2K allowance”];
dividend -> IncomeTax;
salary -> mbalance;
mbalance -> ISA [label=”20K annually”];
}

The Continuing Transformation of the Tech Industry

Assumed Axioms:

  1. Technology has impacted the world in 3 waves (List of companies by founding year)
    1. Hardware – Building the infrastructure
    2. Software – Easily scalable businesses e.g. social media
    3. Hybrid – Combining real-life and assuming always available internet.
  2. Software is eating the world (source) – “More and more major businesses and industries are being run on software and delivered as online services”
  3. Startups and tech in general can be thought as an expanding fractal, reaching all areas of our lives (fractal idea).
  4. The rise of open source can be explained as eating it’s way up the stack as each layer becomes commoditized.

The Past: 1980-1995 – The Transformation of the computer Industry

source: Andrew S Grove – Only the Paranoid Survive

The industry:

  • Had been vertially aligned. i.e. A handful of suppliers, delivered the full end to end package to allow using the hardware and software.
  • A significant change was the standardization of the IBM PC
  • This and other changes allowed shifting to a horizontal marketplace, allowing companies to specialize within each layer and to save costs and earn more profits.
    Microsoft/Intel probably being the biggest winners.

The Present

Any company essentially bundles a bunch of processes/people that allows solving problems within one encapsulated entity, I’ve blogged before about banks as a stack:

The Future

  1. What are the stacks of services that currently deliver value end to end?
  2. Is that market vertical or horizontal?
  3. What model is likely to win under which scenario? You may even see a mix.
    e.g. Apple has vertical integration of iphones, with some horizontal experimentation allowed within the app store, which they then take-over.
  4. You may even find that what wins in the early-stages becomes replaced later.
    e.g. Tableau was excellent software and got deployed at 1000s of companies
    However once it proves the model, the more successful it becomes the easier it may be to replace with an open source alternative.

Prediction: Open Source Eats it’s way up the Pyramid

Stack of oranges in pyramid shape against grey background - CSF015212 -  Dieter Heinemann/Westend61

If you consider most companies as a stack, that software is expanding everywhere and that once it’s used by a significant number of users that an open source version becomes more likely. I think we will see open source companies appear at the bottom of the stack and continue to the top.

Question: What would stop it winning a layer?

Thoughts:

  • A small developer user base. If most users are not programmers, no one will write it.
  • A small user base. Without a wide base to support it, only 1% of users may pay or contribute for open source so it won’t be supportable. In fact if a piece of poor software is introduced it could result in a failed market. i.e. the open product is bad but good enough such that a “better” commercial alternative isn’t viable.

Technology Companies by Founding Year

Steve Case the Third Wave.
https://www.youtube.com/watch?v=u4WS4DhekWU

  • Hardware
  • Pure Software = Overnight
  • Hybrid – Real-Life+Software mixed. Internet integrated sometimes invisibly.

1963 – AMD BIG
1968 – Intel BIG
1974 – FoxConn BIG
1975 Microsoft BIG
1976 – Apple BIG

1977 – Oracle
1984 – Sybase Database, MathWorks Matlab, Cisco, Dell BIG
1985 – Corel Draw
1987 – McAfee virus, Wolfram, Peoplesoft ERP
1988 – Avast Security, Trend Micro Security
1989 – Citrix Remote
1990 – BusinessObjects SAP
1991 – AVG security
1991 – Macromedia Multimedia, Palm PDAs, SUSE linux
1993 – Informatica, Qlik Visualization, Redhat Linux, Tucows Domains, Nvidia
1994 – Amazon, RealNetworks Media
1995 – Cisco Webex, MySQL AB
1996 – Ancestry.com, Taleo HR, Sleepycat DB, Juniper Networks, F5 Networks
1997 – Blackboard Education, Kaspersky Lab Security, TIBCO
1998 – BroadSoft Networking, Symbian OS, VMware VMs, Google BIG
1999 – Apache Foundation, Basecamp, DivbX, Fieldglass, Napster, Salesforce
2000 – Tripadvisor, Mobipocket, Sportradar, Jetbrains Java, zipcar
2001 – Foxit, LumenVox Speech
2002 – Shazam, Linkedin BIG, Meetup, GoPro
2003 – Docusign, LogMeIn, Palintir, ServiceNOW, Splunk, tableau, Skype, Tesla
2004 – APIgee, Canonical OS, SugarCRM, Facebook BIG
2005 – Automattic WordPress, Infobright DB, Mozilla Corp, WorkDay HR, Etsy
2006 – MuleSoft, OpenDNS, Xero Accounting
2007 – Heroku, Lucidworks Search, ZenDesk HR, ZeroTurnaround Java, FitBit
2008 – Balsamiq, Github BIG, airbnb BIG, Twilio
2009 – Grindr, NetObjects, PagerDuty, Okta ID, SendGrid Email, WhatsApp BIG, Uber BIG
2010 – Datadog Monitoring
2011 – Firebase, Hortonworks Data, Zoom Video, Twitch
2012 – G2crowd, Looker, PlanGrid, Instagram BIG
2012 – Databricks, Docker

jq – An open source implementation of q

During this lockdown I was due to take some holidays, originally to visit Pisa with Elaine. Instead of visiting Pisa I took a week off to code, for me it was just as much fun, I’m not sure Elaine agreed. This is the outcome of that week:

jq-language

jq – http://timestored.com/jq/

So far it’s extremely limited, casting, parsing, list definitions and a handful of operations. It has however been insightful. The first 2 days were spent hashing out code to make the pure fundamentals work in any way possible. On the 4th day I began to realise some very verbosely implemented operations could be done in a much simpler way. Then I began to see such savings again and again. Perhaps after the first decade I would have it whittled down to Arthurs two-pager.

An inordinate amount of fun was had when I discovered I could host the application fully in browser  as doppio provides a method of running a full JVM:

jq Online Sandbox – http://timestored.com/jq/

So far it’s useful for basic snippets but I really think such a safe and easily launched environment would be great for onboarding new users to the language.

Quotes that float in my Mind

A colleague asked me to name books or articles that have influenced my thinking (Great question). However my mind works, I seem to store some ideas from multiple places but associated to short quotes. Here’s some of my favourite:

  • You only learn by listening, not by talking.
    • The best thing you can do to make a friend is to really listen to someone.

Software:

  • C.A.R.HoareThere are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult.
  • Leslie Lamport“State the problem before describing the solution”
  • Often when dealing with messy code or 3rd party teams that aren’t delivering or need help, I try to think to myself “be the change” and try to make a difference. A quote commonly attributed to Gandhi. I had one colleague that would jibe back “Don’t be a hero” that the soldier with his head over the parapet gets shot. He felt triumphant when he realised Gandhi was assassinated. I still try to be the change 🙂
  • Tom Cargill – “The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.”. Why I have to take a deep breath when someone says oh that’s just one line of code, rarely if ever is it one line of code. Especially on a legacy system with no tests and a business that specifies unclear demands.
  • There are two types of companies/people. Those..”that sees quality and efficiency as opposing forces, or one that sees them as inseparable“. If you rush to push things out quick now, the overall delivery speed will be slower. – HN post.
  • Fred Brooks – “The bearing of a child takes nine months, no matter how many women are assigned.”
  • Donald Knuth – “premature optimization is the root of all evil (or at least most of it) in programming.” Until you have a benchmark showing where time is being taken and that this area is the constraint we will not be rewriting this currently simple clear code.
  • Brian Kernighan“Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.”
  • – “There is no silver bullet that’s going to fix that. No, we are going to have to use a lot of lead bullets.”
  • Martin Fowler – “If it hurts, do it more often”

Work in General

  • If you accept sh*t and shovel sh*t, don’t complain when people bring you more sh*t to shovel. You brought it on yourself. – James.
  • The best way to challenge something, may not be to oppose it but to ask “why” it’s being done.

Misc

  • The power of habit – “Practice makes Permanent” – Practice by itself will not make you better, you must reflect and get feedback on the practice you’ve done and consider how you can improve.

Java scratchpad / Snippet Runner

I quite often find myself wanting to write quick snippets of java. Similar to little bash scripts.

This usually means I have to:
– launch eclipse
– start a new project
– create a new class
– write public static void main
– try to remember why I started eclipse…..

Well no more. I decided we could do things better. I present:

JPadIt lets you run little snippets of java code and instantly see the result rendered (hopefully) beautifully as either html tables or in a console mode..

Let me know what you think? Do you find it useful? Can we make it better? Definitely!

Apologies but there’s lots of bugs and we need to improve result presentation)

Julia Programming Language

Recently I’ve been looking into Julia, below are my somewhat random jumbled thoughts so far. I’ve also put together a function listing for julia that should help beginners. I should mention upfront that I actually really like the idea of Julia and that it has a lot of promise.

1. The long time taken for packages/recompilation is annoying and it failed on me a few times.

2. It just annoys me that the index starts at 1 – different than most other languages.

3. One of the most successful modules in python is pandas. Why not take this further and make DataFrames a full part of julia i.e. Make nulls, dataframes and DataArray the builtin for all vectors. If julia is going to be for manipulating data, there are always going to be nulls. Why have the duplication between julia arrays and DataFrames, make them one and the same.

4.Has a number of builtin functions that duplicate functionality available in just as short a form:


fill(0.,3)
zeros(3)
ones(3)

They add little and pollute the global namespace. Ken Iverson gives a wonderful talk on reducing a language to the minimal core here:
https://www.youtube.com/watch?v=8kUQWuK1L4w

5. There are many small bugs, inconsistencies and irritations such as:

Should super()/subtypes error on anything? because it does.


julia> super(bb)
ERROR: MethodError: `super` has no method matching super(::Regex)
julia> subtypes(bb)
ERROR: MethodError: `subtypes` has no method matching subtypes(::Regex)
julia> super("2")
ERROR: MethodError: `super` has no method matching super(::ASCIIString)

cd silently fails and isn’t cross platform.


julia> cd("/temp/")
julia> homedir()
"C:\\Users\\DAVE"

It should at least throw an error. Even better would be if julia supported forward slashes regardless of platform. Ensuring julia scripts are cross platform.


run(pipe(`echo world` & `echo hello`, `sort`))

The run command is weird, not like other languages at all. It uses backtick to enclose the command, supposedly to allow easy copy pasting
but what’s the point of that when you cant use piping within a single command, so you will need to break out each command separately anyway. Rather than this string interpolation I could just have used join() myself and have less surprise. This interpolation will also possibly cause issues with user entered commands. (A bit more browsing reveals there is a significant number of people find this a more powerful technique)

Writing to a file isn’t visible. This is very non-intuitive:


julia> f = open("a.txt", "w")
IOStream(<file a.txt>)
julia> write(f, "hello!")
6
julia> readall(f)
""
julia> f = open("a.txt")
IOStream(<file a.txt>)
julia> readall(f)
"hello!"

If possible I’d prefer not to have to even specify “w”, rather when its used by a function,
depending if its read or write, those functions handle the underlying details.

Best Method of Creating Windows Installer for Java Programs.

Requirements

  • Installer that adds link to start menu and file associations to windows
  • Running program appears as an exe and has its’ own file name in windows task manager
  • Use the built in JRE where possible or ask user to download if not available (bundling considered too large)

The finished installer can be seen at http://www.timestored.com/qStudio/

Solution

The best solution I found was launch4j and innosetup

launch4j

Launch4j was the only wrapper I found that let me keep a jar and have multiple exe’s that pointed to the same jar. Allowing me to have two programs but not double the size. One config to keep in mind when doing this is to use the chdir option with . to ensure paths work out properly.

InnoSetup

the tricky part with inno setup was getting the file associations to work. Here’s what I used:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
 
[Icons]  
Name: "{group}\qStudio"; Filename: "{app}\qStudio.exe"
Name: "{group}\sqlDashboards"; Filename: "{app}\sqlDashboards.exe"
 
[Tasks]  
Name: qAssociation; Description: "Associate .q extension"; GroupDescription: File extensions:
Name: kAssociation; Description: "Associate .k extension"; GroupDescription: File extensions:
 
[Registry]
Root: HKCR; Subkey: ".q"; ValueType: string; ValueName: ""; ValueData: "qKDBsource"; Flags: uninsdeletevalue; Tasks: qAssociation 
Root: HKCR; Subkey: ".k"; ValueType: string; ValueName: ""; ValueData: "qKDBsource"; Flags: uninsdeletevalue; Tasks: kAssociation 
Root: HKCR; Subkey: "qKDBsource"; ValueType: string; ValueName: ""; ValueData: "q KDB source"; Flags: uninsdeletekey;
Root: HKCR; Subkey: "qKDBsource\DefaultIcon"; ValueType: string; ValueName: ""; ValueData: "{app}\QFILE.ICO"
Root: HKCR; Subkey: "qKDBsource\shell\open\command"; ValueType: string; ValueName: ""; ValueData: """{app}\qStudio.exe"" ""%1"""

Combining into an ANT script

You want to

  1. Compile your JAR
  2. Create exes for both programs that point to the jar
  3. Place the exe’s into an installer

Like so (you need to install inno setup):

1
2
3
4
5
6
7
 
<launch4j configFile="src/main/resources/launch4j-qstudio.xml" />
<launch4j configFile="src/main/resources/launch4j-sqldashboards.xml" />
 
<exec dir="." executable="${INNO.SETUP.DIR}">
	<arg line="/cc '${basedir}\src\main\resources\windowsInstall.iss'" />
</exec >

Finished Installer

Can be found at http://www.timestored.com/qStudio/

 

Setup java speech jsapi using FreeTTS

Getting JSAPI setup to work using freetts seems to be quite difficult as seen by here, here, here…..

Turns out the install instructions were incomplete. The instructions were:

1. Go to the FreeTTS/lib directory
2. Type .\jsapi.exe
3. If the binary license agreement is acceptable, accept
it by clicking “I Agree”. The jsapi.jar file will be unpacked
and deposited into the lib directory.

What you actually want to do is:

  1. Download FreeTTS
  2. Unzip the freeTTS binary package and check inside the \lib directory for jsapi.exe
  3. Run Jsapi.exe, say yes, to unpack jsapi.jar
  4. Find your JRE directory, mine was C:\Program Files\Java\jdk1.6.0_03\jre\lib\ext
  5. Copy all the Jars (jsapi.jar, freetts.jar, cmu_time_awb.jar, cmu_us_kal.jar, etc.) to that directory
  6. Check in netbeans your projects properties ie right click on project->properties->libraries->manage platforms and the jars should be listed there. If they are not, restart and hopefully they should now be there.
    Netbeans Freetts Setup
  7. Copy speech.properties to the relevant folder. To find out where this is run the HelloWorld example that comes with FreeTTS. In my case the file was located in C:\Documents and Settings\Username\java.home\lib and contained the following
    1
    2
    3
    4
    5
    
    # Modify this accordingly...
    #
    #TextSynthEngineCentral=com.sun.speech.engine.synthesis.text.TextEngineCentral
    #
    FreeTTSSynthEngineCentral=com.sun.speech.freetts.jsapi.FreeTTSEngineCentral

    Or you can set the property using

    1
    
    System.setProperty("freetts.voices", "com.sun.speech.freetts.en.us.cmu_us_kal.KevinVoiceDirectory");

Any problems, post here and I will try to help you out.

Symfony sfGuard – Setting up users,groups,permissions

sfGuard is a Symfony plugin that implements a user management and login system for an application. It supports both groups and individual users… and it saves you from having to ‘roll your own’ user administration system. This guide assumes you have followed the steps given in the readme and that you now want to begin setting up users/permissions etc.

1. Create links in the backend menu to the user/group/permissions tables.

Edit apps/backend/templates/layout.php and add these items to the menu.

1
2
3
<li><?php echo link_to('Users', '@sf_guard_user') ?></li>
<li><?php echo link_to('Groups', '@sf_guard_group') ?></li>
<li><?php echo link_to('Permissions', '@sf_guard_permission') ?></li>

2. Create a login/logout link on the frontend.

Edit apps/frontend/templates/layout.php and add these items to the menu. (Notice the use of $sf_user in the templates.)

1
2
3
4
5
<?php if($sf_user->isAuthenticated()): ?>
   <ul><li><?php echo link_to('Logout', '/logout') ?></li></ul>
<?php else: ?>
   <ul><li><?php echo link_to('Login', '/login') ?></li></ul>
<?php endif; ?>

3. Create some users, groups, permissions

for us to play with using the backend. Create user->basicUser, group->basicGroup, permission->basicPermission. I will be using a basic setup where users always belong to a group and the group has permissions. I will not assigning permissions to individual users. therefore give basicGroup the basicPermission. and you will have something similar to this:

sfGuard user

4. Restricting access to certain modules/actions

Similar to how I never set individual permissions for one user I make it standard that I only ever set permissions using credentials. ie. In the application I never restrict security dependent on user or group id only on permission/credentials. This allows greater flexibility in the future. Note sfGuard gets confusing to some people because many documents talk about credentials, well basically credentials are what is called in sfGuard permissions.

If we have a module called “question”, inside of apps/frontend/modules/question we create a config folder and a new security.yml. Inside of apps/frontend/modules/question/config/security.yml we would have

1
2
3
all:
    is_secure: on
    credentials: basicPermission

To set permissions on an action level we would have something similar to the following:

1
2
3
4
5
6
7
8
9
10
all:
    is_secure: on
    credentials: basicPermission
 
index:
    is_secure: off
 
new:
    is_secure: on
    credentials: basicPermission

Part 2 will detail setting up user registration