23 Dec 2012

Ruby Off The Rails - Ep. 1 -- Zero to Ruby -- Pt. 1

Ruby Off The Rails is a new series of screencasts that I will be publishing. The first cluster of videos will address the problem of getting started with Ruby development by using a virtual installation of Ubuntu Linux running inside Virtualbox. It’s simpler than it sounds and it preserves the integrity of your everyday system. Here’s the first installment, enjoy! (I recommend opening the video for fullscreen display to better view the text)

Ruby Off the Rails - Ep. 1 - Zero to Ruby - Part 1 from ZPH on Vimeo.

Commands from Ep. 1

22 Dec 2012

JDBC, MsSQL, and JRuby, Oh My!

I was recently in a position of needing to access information from a legacy database on a Microsoft SQL Server ( 2005 ). Since I’d prefer that the contact came from a *NIX system (probably Ubuntu), I checked out ODBC and JDBC as options. Based on a couple of blog entries by fellow Rubyists and ODBC users, the ODBC configuration files add an extra layer of complexity for the database connection. I chose to do it with JDBC because it allows those settings to be configured in the script/config itself. Also, with the sqljdbc*.jar running on the JVM we have platform independence. Now, on to the implementation:

  • On the SQL Server box
  • Allow mixed authentication (it appears that JDBC does not interface with Windows User credentials based on cursory research)
  • Setup a unique SQL Server Auth User who has the appropriate degree of authorization on the database. Consider read-only if your application is limited to querying the DB rather than writing to it. This will limit the risk of inadvertent writes.
  • Depending on setup, you may need to open ports 1433 and 1434 on the SQL Server’s firewall. If possible, only do this step on an internally facing instance of the SQL Server. (Less attack surface to the internet is a good idea).
  • Test the connection by using SQL User authentication when connecting with the builtin Microsoft SQL Server Management program. If all goes well, this tells you that it’s configured correctly when accessing from the same system. Next to connect from a remote system. You’ll need the following pieces of information: the database’s name, the database ip address, the database instance name (ie defaults to SQLEXPRESS when using SQL Express), the Sql username and password
  • On the remote system (the Linux box)
  • Download and extract the following file from Microsoft, this is the JDBC adapter file that they provide http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=21599 OR the direct link at the time of this article: http://download.microsoft.com/download/D/6/A/D6A241AC-433E-4CD2-A1CE-50177E8428F0/1033/sqljdbc_3.0.1301.101_enu.tar.gz Extract that into a convenient place in near where your connection code exists. I chose to use a standard Ruby project layout as follows: |-- Gemfile |-- Gemfile.lock |-- Guardfile |-- README.md |-- bin |-- features |-- jars | |-- sqljdbc.jar | |-- sqljdbc4.jar |– sqljdbc_3.0.1301.101_enu.tar.gz |– lib | |– README | |– jruby_demo.rb | |– tds_demo.rb |-- demo.rb |-- spec – spec_helper.rb 7 directories, 16 files Notice that the sqljdbc\*.jar files are inside jars which is on the same level of nesting as the lib directory. This is important when referencing their location in the code. Then install JRuby if not currently installed. I use [RVM](www.rvm.io) for easy Ruby switching. (Thanks to Keith Burnett for posting a comment indicating that this portion was out of order... silly me!) Once the JARs are extracted, runbundle install` to request that Bundler install the required dependencies. Next modify the following script to help you get started. Note that you’ll need to replace the values of the CONSTANTS as appropriate for your IP addresses, username, passwords, etc.Of further note is that a class will need to be defined for each table in the database that you wish to access. I briefly outline this in the comments of the code. This is fairly standard ActiveRecord use. Questions, comments, post them here or to me on Twitter @_ZPH.

18 Dec 2012

Refactoring... a Weighty Question

I’ve been working on a library for easily exporting Trello information to CSV, TSV, and XLSX. It’s currently working well for all three formats although the code itself is first or second draft quality. (Not that XLSX has some issues of its own when done from Ruby. I’d like to solve it eventually but it’s on the back seat since the files still open.) I took the time to import it into CodeClimate’s automated test system to see where potential issues are. It revealed that one particular set of methods was far too complex. These were the methods that generate the CSV or XLSX file. I sat down to refactor them while under the weather this weekend and here’s what I accomplished: -Extracted code out into methods (needs more of this as methods don’t have single responsibilities) -Abstracted the process into a single uber method that calls sub methods (could be private methods later in development) -In order to abstract the process to be identical for CSV and XLSX, I found one spot that needed an eval(content) statement. ( Perhaps in the future I can adapt a lambda to better suit this purpose?? ) Here are the different versions of it: Before-After-Diff-Looking at it again I want to extract the different classes into their own files for better compartmentalizing the code. Let me know in the comments what code smells are present or what refactorings are still advised.

15 Dec 2012

rcodetools and VIM

I’ve been listening to RubyRogues which introduced me to Avdi Grimm from the Ruby community. I’ve also been subscribed to his RubyTapas screencast series which is a 2x a week Ruby show.

He uses a neat tool called rcodetools to execute Ruby code from within an Emacs buffer. Unfortunately, it wasn’t working when I tried to use the VIM counterpart, with segfault codes showing up. Here’s the fix: ( I’m using RVM to manage my Ruby Versions ) *Install Tim Pope’s excellent rvm extension for VIM cd ~/.vim/bundle git clone git://github.com/tpope/vim-rvm.git

*Install rcodetools using RVM gem install rcodetools

*Remove keybindings in other programs (ie Dashboard & Expose) for F10, F11, F12 or rebind the keys per README.vim

Now enter a VIM file and type F12 on any lines that you want the output from followed by F11 to evaluate the whole file. If you want to only re-evaluate the current line use F10 instead of F11.

It’s a great tool for terminal screencasting! Thanks Avdi for letting me know that this tool is available.

26 Nov 2012

Script to Update Sandy Knolls Metes and Bounds Software on OSX

After frequently finding myself not using the current version of Metes and Bounds, I wrote a Ruby script to automagically do that very thing.  It checks current version, downloads the current version, and installs it into /Applications (BEWARNED, it nukes your current version of Metes and Bounds.app, use at your own risk):Thanks to this random blog for the hint on mounting DMGs from commandline!

12 Aug 2012

CodeRetreat @SteelCityRuby: A Post Mortem

My head has finally stopped spinning from the CodeRetreat at this year’s Steel City Ruby Conference (2012), the fires are extinguished at work, and I’m ready to document the experience. I arrived as the first participant, not because I’m overachieving but because the Pittsburg taxi system is very irregular. I allowed the same time as the day before for a cabbie to pick me up, which was 30 minutes. Instead I was swooped up in 10 minutes and arrived nearly an hour early for the CodeRetreat at 4Mom’s office. The CodeRetreat was lead by the charismatic Corey Haines, the same bloke who set a wonderful tone on day one of @SteelCityRuby by emphasizing personal networking. In fact, the tone set by his keynote speech kept the conference largely free from individuals using their laptops during speeches! Quite a difference from other conference scenes that I’ve experienced :). But back to the point, the CodeRetreat was held in the offices of 4Moms and the food was sponsored by those fine folks. We started the day by pair programming Conway’s Game of Life using Test Driven Development. The first round lasted 45 min and we were then told that the specs had changed and to fix our code in another 10 min. Thankfully our code was relatively modular and the changes needed were minimal. We reviewed the amount of code revision needed to implement small requirement shifts and discussed how to build in flexibility and simplicity into the code.

Notes: Revelations: spending time learning social intricacies of collaborating

04 Aug 2012

The Way Conferences Should be - SteelRuby Pittsburgh

I’m attending my first Ruby conference this weekend, YAY!

I want to share a great thing that this conference is doing, which is the emphasis on community and interraction. From the first moment of the conference the organizers recognized that the human capital present is not confined to the speakers. With that in mind the schedule is set for 30 min talks and 30 min ‘hallway sessions’ (ie discussing the topic or discussing Ruby/work/code projects/etc).

So, combine this progressive schedule with a keynote introduction by Corey Haines, where he set the tone for the conference, ‘these are not strangers, these are friends and colleagues that I don’t yet know.’ (my paraphrase is a loose one to the spirit of the talk and I’m also leaving out the cute photos of kittens that filled his slides).

Corey’s talk, formally about how to get the most out of a conference, ended with the encouragement and challenge to have everyone in the conference meet and get to know 20 new people this weekend. Which sounded GREAT until the slides ended and the absolute introverted terror set in. I tend to share very little personal information in my blog, but I’ll say this: I’m introverted and have difficulty breaking the ice myself. That said, I have little to no trouble interracting, conversing, and carrying a conversation in a group like this, after that initial hurdle is over.

Back to the point, I remember looking around and thinking that a presentation like Corey’s was all well and good. I hoped that this conference would be technically interesting and possibly even some interesting social stuff, but I didn’t expect it to feel radically different than my prior confs. It did/does.

People approached me, shook my hand, asked what brought me to the conference. I talked about being a hobbyist at this programming stuff, but a hobbyist who spends much of the last two years working on Ruby & GIS. I introduced myself to all sorts of people and we geeked, it was great. And each initial experience of breaking the ice was still tough and emotionally expensive, but a little less so each time.

Thanks SteelRuby, you’re doing it right.

Thanks fellow Rubyists, I’m glad to be getting into the community.

PS - Putting names to faces with people that are well known in the Ruby community is cool and each and everyone one of them have been awesome and friendly.

15 Jul 2012

My Firefox confuzzled my Nokogiri

I’ve been working on a project to import a large dataset from HTML pages where the content isn’t very clearly divided with CSS classes. The tool that normally works for me is Ruby’s Nokogiri library.

I’d had some trouble in the past where Nokogiri insists that a certain xpath doesn’t exist, while it can clearly find it through the css selector. This time instead of going with the css, I investigated to see why it wasn’t finding the ‘correct’ xpath. After playing with it in a Pry session, I figured it out. Here’s the xpath that Firefox/Firebug/Firepath all report: //html/body/p[4]/table/tbody/tr[3] Here’s the true xpath as parsed by Nokogiri: //html/body/table/tr[3]

Notice that the ‘p[4]’ element disappears along with the ‘tbody’ element. After reading on StackOverflow.com about the issue it sounds like Firefox ‘corrects’ improperly formed HTML by adding tags. So thus the disparity between what Firefox sees (the orderly HTML standards) and what Nokogiri sees (the cold hard real-life web).

Now I want a browser plugin that shows me what is truly there so I don’t have to interpret and modify the information that Firefox gives me. Please speak up in the comments if you know a better way to avoid this annoyance.

28 May 2012

An Exercise: Contact Scraping or How to Get Hired with ZeroMail

I’m trying out ZeroMail and saw their job listing at zeromail.

Though I’m happily and gainfully employed, it looked like an interesting exercise. The challenge is (verbatim from website): (1) Download http://zeromail.com/static/download/emails.txt (2) Write a python program that extracts contact information from signatures in the emails (3) Send your solution to bart@zeromail.com or skype it to bartjellema

After an hour of tinkering with it on my own terms (my Ruby > my Python), I’m realizing it’s more complex than I initially expected.

I see it as a multipart process: (1) Strip junk characters and lines out of the data (2) Determine and locate signature blocks (3) Extract pertinent data from signature blocks (might resort to wordlists of given names)

I tried using easily identifiable information through regexes (ie looking for Australian and US format phone numbers). This yielded decent results on the numbers themselves and I’m considering using phone number’s (or email addresses') line numbers as ‘hotspots’ that I can extract and further process.

If I spend more time on the process, I’ll post my results here.

28 Apr 2012

My Kingdom for a Good Fuzzy Finder

I’ve been working on my VIM configurations recently which involved cleaning out cruft from when I hadn’t transitioned to using Pathogen. While doing so I realized that I wanted to take the advice of DestroyAllSoftware.com and switch away from Nerdtree to something that doesn’t involve a context switch. CtrlP to the rescue! Slick addon that I’m happy to have in my arsenal. Plus, it didn’t necessitate installing and configuring VIM with Ruby 1.8.7 as Command-T did :).