Information – Acodemics

May 23, 2016

The book that I’ve been working on for some time is nearly finished. It is aimed at teaching young adults programming in a more formal setting, each chapter is a single lesson that can be taught over 1-2 hours. There are some draft chapters already available on the learning to program page, the completed chapters will be added here eventually.

The real aim is to produce a set of materials that can be used by students and an additional set of materials aimed at the teachers, including further reading and explanation of what happens in each session as well as model answers and suggested homework.

It will initially be released as an eBook and print copies available separately.

SVN To Git – Advantages and Perks

I recently gave a talk to a group of people looking to move from SVN to Git. I made a presentation discussing the various differences between the two systems and the possible advantages of Git over SVN.

It’s always a hard thing to change something that a developer is comfortable with but I think that by presenting some possible new workflows and features it’s possible for teams to decide for themselves that they wish to change.

The PDF version of the slideshow can be found at this link git SCM Training.

The main take away from the presentation is showing that by using branching as a part of the work flow it opens up totally new ways of collaborating on code within teams; for example by using feature branches a subset of developers can code review a feature before it is merged into master, these things are not available or not recommended with more traditional centralised version control systems.

Python Importing Global Variables – Reference or Value?

March 6, 2015 / Jack Concanon / 0 Comments

I’ve been refactoring a large Python program into separate modules for readability and ease of unit testing and part of that involves moving global variables into a separate globals module. I was interested to see if importing this globals module into other modules and then updating a variable would mean that other modules see the change, here’s what I found –

From globals import *

My first instinct was to use from globals import * in my sub modules, let’s look at an example –

globals.py

VAR = 1

VAR = 1

main.py

from globals import *
from test import *

print "First: %d" % VAR
VAR = 2
print "Second: %d" % VAR

printVar()

from globals import *

from test import *

print "First: %d" % VAR

VAR = 2

print "Second: %d" % VAR

printVar()

test.py

from globals import *

def printVar():
	print "printVar(): %d" % VAR

from globals import *

def printVar():

print "printVar(): %d" % VAR

Running the main.py we get this output –

First: 1
Second: 2
printVar(): 1

Obviously updating the global variable VAR in main.py does not reflect in test.py, this is because from globals import * will import by value giving main.py a local copy of VAR.

Import globals

In order to get a single instance of the global variable VAR we need to modify our program to use import globals and then reference the VAR variable directly inside globals.

Let’s modify the scripts –

main.py

import globals
from test import *

print "First: %d" % globals.VAR
globals.VAR = 2
print "Second: %d" % globals.VAR

printVar()

import globals

from test import *

print "First: %d" % globals.VAR

globals.VAR = 2

print "Second: %d" % globals.VAR

printVar()

test.py

import globals

def printVar():
	print "printVar(): %d" % globals.VAR

import globals

def printVar():

print "printVar(): %d" % globals.VAR

Running this will give us the following output –

First: 1
Second: 2
printVar(): 2

Much better! The solution to this problem is to update the VAR variable directly inside globals allowing updates to be seen from both modules that import globals.

Bag of Words Implementation

February 17, 2015 / Jack Concanon / 0 Comments

Bag of words is a method for reducing natural text into a representative model for use with machine learning and natural language processing.

I used it to train a network on log messages and then assign scores to known log messages, based on a bag of words representation the neural network can give a score of how well the test data matches.

class BagOfWords:

    def __init__(self):
        self.dict = {}
        self.i = 0

    def addVocab(self, line):
        for word in line.split(" "):
            if word not in self.dict.keys() and not word.isdigit():
                self.dict[word] = self.i
                self.i += 1

    def vocabScore(self, line):
        score = [0] * len(self.dict.keys())
        for word in line.split(" "):
            if word in self.dict.keys() and not word.isdigit():
                score[self.dict[word]] += 1
        return score

class BagOfWords:

def __init__(self):

self.dict = {}

self.i = 0

def addVocab(self, line):

for word in line.split(" "):

if word not in self.dict.keys() and not word.isdigit():

self.dict[word] = self.i

self.i += 1

def vocabScore(self, line):

score = [0] * len(self.dict.keys())

for word in line.split(" "):

if word in self.dict.keys() and not word.isdigit():

score[self.dict[word]] += 1

return score

Image Recognition Tool

August 13, 2014 / Jack Concanon / 0 Comments

As I’ve recently got quite into machine learning tools I’ve written a small GUI based tool that uses the PyBrains library to ‘learn’ a common theme from a folder of training data. To test new images, drag and drop them onto the tool to get a percentage of similarity to the test data.

The tool is written using Python, PyBrains, Glade2 and PyGTK so you need those libraries available.

The tool can be downloaded here – image_recognition_tool

August 4, 2014

A presentation I made for work that is an introduction to machine learning. Notes included with power point slides. Machine Learning

Reblog: Python Minidom and Whitespace

January 17, 2014 / Jack Concanon / 0 Comments

A good tutorial providing fixes for printing of XML using Python and Minidom. Specifically fixing added whitespace and the conversion of characters into HTML entities.

Unit Testing and Code Coverage Results in Jenkins CI

August 16, 2013 / Jack Concanon / 0 Comments

Unit testing is great, code coverage is great and continuous integration servers are great; but what isn’t great is combining all three into something useful!

If you are coding in Java then code coverage is fantastically simple but what if you use C++ and GoogleTest and want the same level of graphification? Luckily it is also simple if you know how – so you’re in the right place.

The scenario here is that you have a C++ project being built on a Jenkins/Hudson continuous integration setup and you also use Google Test for your unit tests. The software required for this is GCC 4.7 which should contain a tool called gcov, this will create the actual coverage files for your executable but will require some additional linker and library options to be added to your build scripts or make files. The final piece of software required is called gcovr, this is a Python script that is designed to read the output files from gcov linked executables and convert it into an XML format that can be read by the Cobertura plugin in Jenkins/Hudson.

It is a good idea to add a separate make target for coverage as it requires the executables to be built with no optimisation and a lot of debug symbols, so not very useful for production environments.

Add the following to your CPPFLAGS –

-fprofile-arcs -ftest-coverage -O0

1	-fprofile-arcs -ftest-coverage -O0

And this to your LDFLAGS

-lgcov

-lgcov

So for example you could have a section in your makefile that looks like this (assuming the test target builds your unit test.) –

coverage: CPPFLAGS += -fprofile-arcs -ftest-coverage -O0
coverage: LDFLAGS += -lgcov
coverage: test

coverage: CPPFLAGS += -fprofile-arcs -ftest-coverage -O0

coverage: LDFLAGS += -lgcov

coverage: test

After you have run the coverage build target you then have to run the test executable, this generates .gcov files that describe which lines of code are covered and by what functions.

Then ensure that you can access the gcovr script and run it with the following arguments –

./test
chmod a+x ./gcovr
./gcovr --gcov-exclude=".*gtest.*" -x -d &gt; coverage.xml

./test

chmod a+x ./gcovr

./gcovr --gcov-exclude=".*gtest.*" -x -d > coverage.xml

The –gcov_exlcude will remove any unnecessary parsing of files from the GoogleTest framework. The output is placed into the coverage.xml file which can then be accessed from Jenkins by adding a new Post-Build Action from the job configuration page.

Unit Testing C# With Data in Visual Studio 2012

May 16, 2013 / Jack Concanon / 0 Comments

When writing unit tests it is likely you will need to include some additional input files containing data or settings etc. If you’re trying to include some data for a C# unit test and are using Visual Studio 2012 follow this guide –

Add a separate folder for data to the project then ‘add an existing item’ and select your files.
In the Solution Explorer, navigate to your newly added file, right click and go to properties.
Set ‘Copy to Output Directory’ to an appropriate option.
When you want to use this data in a unit test add the following before the function – [DeploymentItem(“Data”, “Data”)]

The line in the last step will copy all data from Data in your build directory to the a new folder in the test directory.

I presume that these instructions will apply to older versions of Visual Studio.

Categories

Meta

CategoryInformation

SVN To Git – Advantages and Perks

Python Importing Global Variables – Reference or Value?

From globals import *

globals.py

main.py

test.py

Import globals

main.py

test.py

Bag of Words Implementation

Image Recognition Tool

Reblog: Python Minidom and Whitespace

Unit Testing and Code Coverage Results in Jenkins CI

Unit Testing C# With Data in Visual Studio 2012

Recent Comments