New project, Python, and graphing

Over the last few months, I have found myself becoming interested in data analysis and mining. I recently picked up a copy of O’Rielly’s Data Analysis with Open Source Tools. Flipping through it, I discovered that a lot of the math was above my skill level. I had no problems with understanding how the code snippets worked, but I was clueless when it came to seeing the equations. I decided to that the best way to resolve this was to learn a little about data analysis, write some code related to it so I can understand the connection between the two, and then write a blog post on it.

For this week, I decided to focus on the basics, what I already knew. I decided to start creating graphs, especially line graphs, in the programming language Python. I read up on some tutorials to create line graphs in Python, as well as taking a crash course in Python itself. After working through the tutorials and feeling comfortable with the language, I wanted a practical application to write the code around. I bought a used car about two years ago and each time I filled up the tank, I recorded how much gas I put in the tank and how many miles since I last filled up. This gave me a decent sized data set (and I only entered the last 100 records) to graph. The data was entered using LibreOffice and then saved as a CSV file in the format of gallons,miles. The python script then read in the records, split it on the comma, removed the newline character, and finally calculated the miles per gallon to be graphed. After the miles per gallon was calculated for each entry, it was added to a temporary value that was then divided by the number of records in the data set. This created an average miles per gallon over the entire data set which allowed the user to see how a single fill up did compared to the average. Once this data was created, I was able to pass it to the pylab module in Python and generate a graph of the individual MPG and the average MPG.

The code, which can also be found on my github page, is shown here:

#!/usr/bin/python
# Script reads in a file called data.csv
# Layout:
# gallons_of_gas,miles_between_fillups
# Output is a line graph for between 0 and 50MPG

import numpy as np
import pylab as pl

gallonsList = []
milesList = []
mpg = []
count = []
overallAveMPG = []
totalMPG = 0

file = open('data.csv', 'r')

for line in file:
	split = line.find(',')
	gallonsList.append(line[:split])
	milesList.append(line[split+1:len(line)].rstrip())

itemCount = len(gallonsList)

for x in range(0, itemCount):
	mpg.append(float(milesList[x]) / float(gallonsList[x]))
	count.append(x+1)
	totalMPG = totalMPG + mpg[x]

for x in range(0, itemCount):
	overallAveMPG.append(float(totalMPG) / float(itemCount))

# Create the points and lines
MPG, = pl.plot(count, mpg, 'r')
averageMPG, = pl.plot(count, overallAveMPG, 'g')

# Generate a title
pl.title('MPG of car')

# Label the X and Y axis
pl.xlabel('Fillup number')
pl.ylabel('MPG')

# Create the limits of the axis'
pl.xlim(0.0,float(itemCount))
pl.ylim(10.0, 100.0)

# Create the legend
pl.legend([MPG, averageMPG], ('MPG for each fillup', 'Average MPG'), 'best', numpoints=1)

# write the graph to a file
pl.savefig('mpg.pdf')

I hope in the near future I will change the average MPG line to show how each fill up at the gas station affects the average.

I also am planning on posting about once a week for awhile now, so if you are curious about what I am doing, feel free to comment below or find me on IRC as daemoneye on freenode.

ctags, taglist, and vim

The other day I flipped through a book entitled “Practical Vim: Edit Text at the Speed of Thought” by Drew Neil and stumbled across a section on ctags. I learned how Vim interfaces with ctags and how they would be a valuable asset for larger code projects. A few examples would be:

* many files which contain classes/functions
* more than three or four small files
* single file with over 500 lines of code

To get started working with ctags I ran the ctags command with a few parameters: files to be indexed and any options I want. In my case I created a ctags config file located in my home directory: ~/.ctags. In it are the following options:

–recurse=yes
–tag-relative=yes

The –recurse parameter tells ctags to look at all the files in the subdirectories of the current directory or the directory supplied when the command is run.

The –tag-relative parameter tells ctags to include the directory containing the tags file.

I usually work in the source directory of the project I am working on and run “ctags .” which creates the tags alongside the source code. The format is the tag name, the tag file, then the command to take the cursor to that location. It also has a few special flags at the end that I do not know the meaning of at this point. The file is tab delimited. The output of ctags when I ran it across a project I am working on, this is what the file looks like:

!_TAG_FILE_FORMAT	2	/extended format; --format=1 will not append ;" to lines/
!_TAG_FILE_SORTED	1	/0=unsorted, 1=sorted, 2=foldcase/
!_TAG_PROGRAM_AUTHOR	Darren Hiebert	/dhiebert@users.sourceforge.net/
!_TAG_PROGRAM_NAME	Exuberant Ctags	//
!_TAG_PROGRAM_URL	http://ctags.sourceforge.net	/official site/
!_TAG_PROGRAM_VERSION	5.9~svn20110310	//
ITEM_H	item.hxx	2;"	d
RECIPIE_H	recipie.hxx	2;"	d
addServings	item.cpp	/^void item::addServings(int add)$/;"	f	class:item
barcode	item.hxx	/^		long long int barcode;$/;"	m	class:item
calories	item.hxx	/^		int calories;$/;"	m	class:item
caloriesPerServing	recipie.hxx	/^		int caloriesPerServing;$/;"	m	class:recipie
checkItemCount	recipie.cpp	/^int recipie::checkItemCount()$/;"	f	class:recipie
createRecipie	recipie.cpp	/^void recipie::createRecipie(vector <item> *allItems, string name)$/;"	f	class:recipie
display	item.cpp	/^void item::display(void)$/;"	f	class:item
display	recipie.cpp	/^void recipie::display()$/;"	f	class:recipie
displayData	main.cpp	/^template <class T> void displayData(vector<T> *data)$/;"	f
fillData	main.cpp	/^template <class T> void fillData(vector<T> *data, string filepath, string logFile)$/;"	f
generateList	main.cpp	/^void generateList(vector<item> *items)$/;"	f
getBarcode	item.cpp	/^long long int item::getBarcode(void)$/;"	f	class:item
getCalories	item.cpp	/^int item::getCalories(void)$/;"	f	class:item
getCsv	item.cpp	/^string item::getCsv(void)$/;"	f	class:item
getCsv	recipie.cpp	/^string recipie::getCsv()$/;"	f	class:recipie
getMinServings	item.cpp	/^int item::getMinServings(void)$/;"	f	class:item
getName	item.cpp	/^string item::getName(void)$/;"	f	class:item
getName	recipie.cpp	/^string recipie::getName(void)$/;"	f	class:recipie
getServingCount	item.cpp	/^int item::getServingCount(void)$/;"	f	class:item
getServingType	item.cpp	/^string item::getServingType(void)$/;"	f	class:item
getServingsInDish	item.cpp	/^int item::getServingsInDish(void)$/;"	f	class:item
getServingsPerContainer	item.cpp	/^int item::getServingsPerContainer(void)$/;"	f	class:item
item	item.cpp	/^item::item(void) : barcode(0), name(""), servingType(""), servingsPerContainer(0), minServings(0), calories(0), servingCount(0), servingsInDish(0) {}$/;"	f	class:item
item	item.hxx	/^class item$/;"	c
itemList	recipie.hxx	/^		vector <item>itemList;$/;"	m	class:recipie
logMe	main.cpp	/^void logMe(string msg, string file)$/;"	f
main	main.cpp	/^int main()$/;"	f
makeDish	recipie.cpp	/^void recipie::makeDish()$/;"	f	class:recipie
menu	main.cpp	/^void menu(vector<item> *items, vector<recipie> *recipies)$/;"	f
minServings	item.hxx	/^		int minServings;$/;"	m	class:item
name	item.hxx	/^		string name;$/;"	m	class:item
name	recipie.hxx	/^		string name;$/;"	m	class:recipie
newItem	main.cpp	/^void newItem(vector<item> *items)$/;"	f
newRecipie	main.cpp	/^void newRecipie(vector<item> *items, vector<recipie> *recipies)$/;"	f
orderMore	item.cpp	/^bool item::orderMore(void)$/;"	f	class:item
recipie	recipie.cpp	/^recipie::recipie() : name(""), servings(0), caloriesPerServing(0) {}$/;"	f	class:recipie
recipie	recipie.hxx	/^class recipie$/;"	c
reduceCounts	recipie.cpp	/^void recipie::reduceCounts()$/;"	f	class:recipie
removeServings	item.cpp	/^void item::removeServings(int remove)$/;"	f	class:item
saveData	main.cpp	/^template <class T> void saveData(vector<T> *data, string filepath, string logFile)$/;"	f
servingCount	item.hxx	/^		int servingCount;$/;"	m	class:item
servingType	item.hxx	/^		string servingType;$/;"	m	class:item
servings	recipie.hxx	/^		int servings;$/;"	m	class:recipie
servingsInDish	item.hxx	/^		int servingsInDish;$/;"	m	class:item
servingsPerContainer	item.hxx	/^		int servingsPerContainer;$/;"	m	class:item
setBarcode	item.cpp	/^void item::setBarcode(long long int tmp)$/;"	f	class:item
setCalories	item.cpp	/^void item::setCalories(int tmp)$/;"	f	class:item
setCaloriesPerServing	recipie.cpp	/^void recipie::setCaloriesPerServing()$/;"	f	class:recipie
setMinServings	item.cpp	/^void item::setMinServings(int tmp)$/;"	f	class:item
setName	item.cpp	/^void item::setName(string tmp)$/;"	f	class:item
setName	recipie.cpp	/^void recipie::setName(string tmp)$/;"	f	class:recipie
setServingCount	item.cpp	/^void item::setServingCount(int tmp)$/;"	f	class:item
setServingType	item.cpp	/^void item::setServingType(string tmp)$/;"	f	class:item
setServingsInDish	item.cpp	/^void item::setServingsInDish(int tmp)$/;"	f	class:item
setServingsPerContainer	item.cpp	/^void item::setServingsPerContainer(int tmp)$/;"	f	class:item
setVars	item.cpp	/^void item::setVars(long long int code)$/;"	f	class:item
setVars	item.cpp	/^void item::setVars(string data)$/;"	f	class:item
setVars	recipie.cpp	/^void recipie::setVars(string tmp)$/;"	f	class:recipie
steps	recipie.hxx	/^		vector <string> steps;$/;"	m	class:recipie
update	item.cpp	/^void item::update(long long int)$/;"	f	class:item

Once the tags file is created it is easy to open the files in Vim. Each file gets its own tab with the parameter “-p”, so on the command line: “vim -p *.file-ext”. To load the tags file, we type “:set tags?” in Vim command mode. If the tags file exists in the current directory it can be automatically loaded if we add “set tags+=tags;” to our ~/.vimrc file.

To move about the loaded tags file we go to a function call or class name and press “”, which moves the cursor to the start of that function or class. “” will take us back to where we were. If we want to go to a different function name or class name than where our cursor is currently located we type “:tag {keyword}” and the cursor will jump to that place. There are several other commands that can be used that I have not explored yet so, for now, I will not comment on those.

In order to navigate the tags I found taglist.vim on github and installed it in my ~/.vim directory. I added the line “filetype plugin on” to my ~/.vimrc file. To list each function name for
each open file I run “:TlistOpen” in command mode.

I also added “set mouse=a” within my ~/.vimrc file in order to add mouse support. I can scroll through the list and double click on a function or class and it will take me there as if I did “:tags {keyword}” from command mode.

TDD and Software Development Thoughts

I have been writing code for several years.  Most of it had been for school, but for the last year or so I have been doing more projects for work.  I have also started to write code in my free time to learn more about software development.  Today I went to a coding event, where the goal was to teach others, learn from others, and have fun.  There was a lot of emphasis on paired programming.  The event is something that is held in a lot of areas, where teams of two people work for one hour on one project, usually Conway’s Game of Life.  On occasion, like the one I went to today, we did not do Conway’s Game of Life, but we had to choose from  Cyber Dojo, which is a free place to practice programming.  Then at the end of hour, the code is deleted and everyone finds a new partner.  Today, I mostly learned about test driven development, which I hope to incorperate more into my code.  I also found the social aspect of working with another person to solve a problem is a huge help.

Within the event, we also got to tried remote pair programming.  The common problems I heard were issues with getting software up and running, even though the most common tools were Skype, Google Hangouts, and TeamViewer.  There were some issues with using different Operating Systems, the network cutting out on people, or the application using too much CPU.  I am curious to know what tools others use out in the industry for remote pair programming.  I am also curious what issues that you have had and what was enjoyable about that tool.

Cooking

    I have recently been trying to improve my diet, as part of an effort to lead a healthier life for myself.  I use to cook a fair amount growing up (and was a pretty skinny kid too).  However I started to gain weight when I started college, mostly because I felt like I no longer had the time to make a good meal.  Now that I have a decent job (and honestly, no college degree yet), I decided to take a year off from school and get my health back under control.  I started simple with stir-frys, but I am starting to make more complex meals such as quiche.
    Today along with making quiche for the first time on my own, I decided to experiment a bit with the stir-fry I have been making a lot.  Usually, I use a little vegetable oil or a little butter to grease the fry pan, but today I didn’t have either.  I was trying to think of what I could use as a replacement and decided on soy sauce.  I don’t know why I chose soy sauce other than I had a lot of it. It had more flavor than water (which I have seen done before), and I didn’t have to drive to the grocery store to get it.  On the best case scenario at the time, I decided it would be just like using oil or a little butter, and at the worst case scenario, it would be a flop and I would have to spend some time cleaning the mistake up.  Thankfully it was not the worst case, but it also wasn’t the best case.  It ended up being better than I expected in that it added a little flavor to everything.
    I hope to spend some time to read through ‘The 4-Hour Chef: The Simple Path to Cooking Like a Pro, Learning Anything, and Living the Good Life‘ by Tim Ferriss.  I got the book late last year as a reminder that I need to be cooking more and hopefully as a source of ideas for my own cooking.  I have looked through bits and pieces of it in the past and it looks like it has some really good ideas in it.  I am going to try to take it one day at a time from here on out and avoid processed food as much as I can.

Every Day Carry (EDC)

I have been working on building myself an urban every day carry (EDC) bag and I have decided to share what I carry with me and I would love input about what I should add or remove. This is a list that goes along with what I carry on my person, which is a knife, wallet, keys, notebook, and pens. I have found that altoids (http://shop.altoids.com/) tins are really nice for carrying small items and have three to four of them on me for holding various things. So here is the list:

 

  • Laptop

  • Power Supply for the laptop

  • Glasses/case

  • SOG Multi-tool

  • Two pens

  • Pad of paper

  • Knife

  • Make Magazine Minitronics Survival Pack with solder and heat shrink

  • Pencil Torch

  • Altoids Tin with:

  • Ear plugs

  • Headphone Split

  • 8GB Flash Drive

  • Leatherman Squirt ES4

  • Altoids Tin with:

  • Gloves

  • Tweezers

  • Nail Clippers

  • Chap stick

  • Band-aid

  • Tool kit with:

  • Alligator Clip

  • Mirror

  • Small magnet

  • Large magnet with LED

  • Deck of cards

  • Blacklight

  • 4x AA batteries

  • 4x AAA batteries

  • USB Wall plug

  • Retractable cables:

  • 2x USB to USB Micro

  • USB to USB mini

  • USB to USB Type B

  • Ethernet

  • Male to Male audio

  • HDMI

  • VGA

  • Change/Cash

  • Flashlight

Introduction

This is a blog about things that I find important in my life, things I have discovered, learned, and thought about. I hope to write often, but will be limited to when I have the time. I spend a lot of time writing code, learning algorithms, traveling, and thinking about various aspects of life.

 

I am currently a college student and taking a year off from formal education to travel, learn new things, explore things taught in school that I want to understand better, and pursue hobbies I have.

Follow

Get every new post delivered to your Inbox.