Simple git commit auto fill

Back in the days when I worked for RedHat I remember having a neat autocomplete git hook that filled in my git commit message. I liked the way it was formatted and I remember it being something similar to this.

* /Path/To/Changed/File0 (Method0):
(Method1)
(Method2)
(Struct)
* /Path/To/Changed/File1 (Method0):
...

Here I try to reproduce this effect by having a very simple script that uses the git diff. Here is the script:

git diff --cached | \ [ruby-2.7.1p83]
grep -e '^@@.*@@.*(.' -e '^@@.*@@..struct.{' -e '^diff ..git' | \ sed 's/^@@.*@@ /@@ /' | \
sed 's/(.//' | \
sed 's/@@.struct/@@@ struct/' | uniq | \
awk -F " " '/^diff --git/{gsub(/^./,"",$3);print "#*"$3" :"} /^@@@ struct/{gsub(/{/,"", $3);print "#("$3") :"} /^@@ /{print "#("$2") :"}'

The `grep` line selects the relevant git diff lines. The `sed` lines remote the text that goes around the lines containing the “@@” string. And finally the awk line formats three types of lines: 1. the file change, 2. the method change, and 3. the struct change.

This script has some shortcomings:

  1. New methods, structs do not show up
  2. Sometimes the diff will have a method, struct name that is not actually changed.

Still I’ll make this post as a reminder to myself and a place where I can start next time I want to “give this a shot”

Posted in Uncategorized | Tagged , , , | Leave a comment

Phenology Detection At IBIMET

I was a visiting researcher at the Institute of BIoMETeorology (IBIMET) in Sassari Italy in the month of July 2013. My work there was directed at applying known research from the Richardson lab for using color responses to characterize phenological behavior. You can find the relevant research in this paper.

Research at IBIMET centers on monitoring phenological behavior of the different plant species in the north-west of Sardinia in the Porto Conte – Capo Caccia Nature reserve. The people at IBIMET-Sassari have installed a pan-tilt-zoom camera in a location of interest and have collected images during several months of the species growing in the natural reserve. I have compiled these images in a this movie which shows, among other things, the flowering of a species (around the 8th second). This is picked up by the excess green signal calculated with the Richardson paper concepts.

The gist of my work in Sassari can be summarized in the following figure:

The blue line represents the "raw" signal from the Excess Green Color space on the image series. The red represents the fitted Sigmoid signal. The date is calculated using the inflection point of the sigmoid and is when the flowering occurred.

The blue line represents the “raw” signal from the Excess Green Color space on the image series. The red represents the fitted Sigmoid signal. The date is calculated using the inflection point of the Sigmoid and is when the flowering occurred.

We see how the color signal can pick up relevant phenological variations. The idea is to automate the whole process and have an automata go through large amounts of image databases trying to detect these types of behavior.

Posted in Uncategorized | Tagged , , | Leave a comment

Leaf Counting and Identification.

Automating plant phenotyping. We have added an initial approximation to leaf counting and identification. This will detect the leaf centers and give the number of leafs of a plant. Inspiration came from a paper presented at CVPPP20143-D Histogram-Based Segmentation and Leaf Detection for Rosette Plants. Jean-Michel Pape et. al. 2014. And while there is still some false negatives that need to be addressed, I think the use of a simple distance map, together with local maximum is a pretty good starting point. Here is an example image from our data set:

The circles mark the places where we detect a leaf.

The circles mark the places where we detect a leaf. Notice how the plants in the right did not get detected, this is due to segmentation errors.

The circles mark the detected leaf centers. Some of the leafs of these plants are not detected because they don't translate into a detectable maximum in the distance transformation.

The circles mark the detected leaf centers. Some of the leafs of these plants are not detected because they don’t translate into a detectable maximum in the distance transformation.

Image | Posted on by | Tagged , , , , , , , | Leave a comment

Automatic Season Detection

It’s not very difficult for a human to know when a season starts or ends from seeing an image series. It becomes very tedious to do it when you have millions of images to go through. This is where automatic season detection can aid efforts in academic (e.g. research on global warming) and industrial (e.g. Agriculture) contexts.

Posted in Uncategorized | Tagged , , , | Leave a comment

Image Recognition and Machine Learning are Cool/Scary :)

Check this link out. Cool because emotions (smile == happy), age, identity and lots more can now all be gathered automatically but its scary because emotions, age, identity and lots more can all be gathered automatically.

Check out the Smart Me Up company web page. It shows a really cool way of presenting all these concepts as well as how far these fields have come.

Posted in Uncategorized | 1 Comment

Pan, Zoom with Qt

This is a followup post for my previous pan/zoom post. It is so difficult to find a proper widget that implements Zoom with mouse wheel and pan with CTRL+LeftClick. IMO, it is something that is necessary when viewing an image. I finally found a QT implementation that was fast and simple to understand. Here is the link to the Stack Overflow discussion. Hopefully this post will increase the likelihood of ppl finding a good solution in QT for this issue.

Posted in Uncategorized | Leave a comment

Sound from image

What will they think of next? This awesome MIT project recovers sound from a high-speed video feed. It’s awesome, scary and interesting all at the same time :)

 

Posted in Uncategorized | Leave a comment

Awesome Segmentation for Arabidopsis Thaliana

In the past months I have been working with a project which has the purpose of addressing high throughput phenotyping. We recently had a very successful test that I want to share as a video. What we are trying to do is extract plant data automatically from individual images. Here is the video :)

 

Posted in Uncategorized | Tagged , , , , , | Leave a comment

Day to Day Danish Resident

In Denmark, it’s probably intuitive to say:

  1. We consume less energy on warm days between 0100 and 0300 hours.
  2. We consume more energy cold days between 1800 and 2000 hours.
  3. As temperature drops, the consumption increases. This increase is marked at 5 to 10 degrees.

But it’s really cool to watch it in 3D. Click on the image to get a 3D figure that you can play with :)

Consumption

Plot depicting Temperature vs Hours of the Day vs Electricity Consumption.

Posted in Uncategorized | Tagged , , , | Leave a comment

Getting Reliable Temperature Readings

I wanted to correlate temperature to household energy usage in Denmark. To do this, I needed to acquire reliable temperature measurements from Zealand but it has turned out to be a very hard thing to do.

  1. I logically began my journey at the Danish Meteorological Institute (DMI) but I could not find raw data that I could use. There was a lot of very useful information everywhere, but no historical data that I could download in CSV file (or in any other format).
  2. European Climate Assessment and Dataset (ECA&D): This was my first successful hit where I was able to acquire daily temperature means from a station called LANDBOHOJSKOLEN. This was a good beginning, but I’m looking for measurements that are taken every hour.
  3. Weather Underground (WU): In my search for a finer granularity (hours means instead of day means), my search has taken me to Weather Underground. They seem to have the temperature data measured at 30 minutes intervals, which was really good for my purposes. But they round the temperature values to the nearest degree which might introduce some in the correlation. In any case, I used the “comma separated file” functionality (Here is map of stations) to download temperature data by automating the process with a python script (I might have made it a bit too complicated, but it works).
import urllib2
import os.path
import datetime
import random
import time

WEATHER_PLACE="http://www.wunderground.com/history/airport/EKRK/%s/%s/%s/" \
 + "/DailyHistory.html?format=1"
WEATHER_PLACE="http://www.wunderground.com/history/airport/EKRK/%s/%s/%s/" \
 + "DailyHistory.html?req_city=NA&req_state=NA&req_statename=NA&format=1"
WEATHER_UNITS="http://www.wunderground.com/cgi-bin/findweather/getForecast?setunits=english"
MAX_WAIT_TIME=1 #in seconds.

def dwGenerateDateDict( D1, D2 ):

 # Initialize dateDict
 numdays = abs(D1 - D2).days
 dateList = [ D2 - datetime.timedelta(days=x) for x in range(0,3) ]
 dateDict = dict(zip(dateList, [""]*len(dateList)))

 while ( len(dateList) > 0 ):
 print("Days left: " + str(len(dateList)))

 # Wait for a random amount of seconds
 secs = random.randint(0,MAX_WAIT_TIME)
 print("Waiting for " + str(secs) + " seconds.")
 time.sleep(secs)

 # Select a random date
 ranInd = random.randint(0,len(dateList)-1)
 rd = dateList[ranInd]
 del(dateList[ranInd])

 # Get the csv weather for the date
 urllib2.urlopen(WEATHER_UNITS)
 urlRes = urllib2.urlopen(WEATHER_PLACE%(rd.year, rd.month, rd.day))

 # Read rest into dictionary.
 dateDict[rd] = urlRes.read()
 print("Date : " + rd.strftime("%Y%m%d") \
 + ". Read : " + str(len(dateDict[rd])) + " Bytes")

 return (dateDict)

def dwGenerateCSVFile ( dateDict, filename ):

 fd = open(filename, 'a')
 keys = dateDict.keys()
 keys.sort()
 for date in keys:
 datestr = str(date)
 datedata = dateDict[date]
 timetype = datedata[1:datedata.find(",")] # CET or CEST
 # Remove first list of header
 datedata = datedata[datedata.find("
\n") + 7:]
 # Last separator to front
 datedata = datedata[-7:] + datedata[:-7];
 datedata = datedata.replace("<br />\n",
 "\n"+datestr+","+timetype+",")
 fd.write(datedata)
 fd.close()

if __name__ == "__main__":
 D1 = datetime.date(2011,9,27)
 D2 = datetime.date(2013,9,23)
 DD = dwGenerateDateDict(D1,D2)
 dwGenerateCSVFile(DD, "UnderGroundWeatherRoskildeAirport.csv")

I’m going to start trying different weather APIs and see if any of them can give me the historical information that I need with the granularity level that I want.  More to come…

Posted in Uncategorized | Tagged , | Leave a comment