Raspberry Pi, Raspian Debian Wheezy, Boot Single User Mode

In short, eject the SD card from your Pi, mount it somewhere, and add "single" to the end of cmdline.txt.  I was able to stick the card in my Mac and edit the file.  Eject the card from that computer, stick it back in the Pi and boot it up.  As is typical on Linux boxes it will boot to single user mode and prompt you for the root password.

Once you've booted up and fixed whatever needs fixing you can edit /boot/cmdline.txt on a running Pi and remove the "single" keyword.

Reporting Input Errors from a Rails Model

There are several places outside of the standard validations where you might process user input in a Rails model and want to inform the user that they supplied bad data.  It is not immediately obvious though how you get useful error messages back to the user.  Rails gives your model an instance of ActiveRecord::Errors called errors, which is the standard way to return validation errors.  You have to think of any other user input checking as validation and somehow get error messages into the errors object.

Without further ado, from easiest to hardest:

Standard Validations

With a standard validation Rails handles adding an appropriate error message to errors for you.

class Thing < ActiveRecord::Base
  validates :name, presence: true
end

Custom Validations

To quote from the Rails guide to custom validation:

"You can also create methods that verify the state of your models and add messages to the errors collection when they are invalid."

class Thing < ActiveRecord::Base
  validate :custom_validator
  def custom_validator
    errors[:name] << 'This is a custom validation error on name'
  end
end

Before Validation Callback

Rails provides callbacks at a number of stages of model processing.

To quote from the Rails guide regarding halting execution in a callback:

"If any before callback method returns exactly false or raises an exception, the execution chain gets halted and a ROLLBACK is issued; after callbacks can only accomplish that by raising an exception."

"Raising an arbitrary exception may break code that expects save and its friends not to fail like that. The ActiveRecord::Rollback exception is thought precisely to tell Active Record a rollback is going on. That one is internally captured but not reraised."

It doesn't mention that before_validate callbacks have another option.

Specialized validation should normally go in a custom validator, but if for some reason you have to perform a check in a callback then any time you are processing user input you should do that in a before_validate callback.  Using before_validate allows you to use the standard errors object, which behaves just like it does in a custom validator.

class Thing < ActiveRecord::Base
  before_validation :bv
  def bv
    errors[:name] << 'This is a before validation error on name'
  end
end

Virtual Attribute

You can define virtual attributes in a model.  As shown in the RailsCast you can use custom validation for simple cases.  But some virtual attributes may be doing complex processing that causes you to discover errors in the user input inside the virtual attribute method.  Assignment methods are called before validation.  They're even called before the before_initialization callback for any virtual attributes you pass via new/create.

Apparently this is too early in the process to use the standard errors object, anything added to errors is ignored.  But since these methods execute before validation we can just save up any errors and insert them into the standard errors object during the validation stage.

class Thing < ActiveRecord::Base
  def virtual=(virtual_value)
    @virtual_errors = {name: ['Virtual is invalid']}
  end
  before_validation :bv
  def bv
    @virtual_errors.each do |k,v|
      v.each { |e| errors[k] << e }
    end
  end
end

Rails, simple_form, Twitter Bootstrap, and append

I wanted to put a little delete button at the end of a form field.  We start with the standard simple_form syntax for a form field.

  <%= f.input :name %>

The generated HTML has the form elements inside a set of nested divs which have the classes necessary for Bootstrap styling.

The result looks like:

If we want our delete button to appear on the same line as the form field we need to get it inside those divs.  The Wrapping Rails Form Helpers section of the simple_form documentation explains that you can pass a block to the input method, and some of the developers mentioned in ticket comments on Github that this can be used to add content near the form field.  There seems to be some discussion about improving the syntax so that you don't have to list the name of the model attribute twice, but for now that is necessary.

  <%= f.input :name do %>
    <%= f.input_field :name %>
    <%= button_tag content_tag(:i, nil, class: 'icon-remove'), type: 'button', class: 'btn btn-danger' %>
  <% end %>

The result looks like:

Looking good.  But a review of the Bootstrap documentation for forms turns up their "Append with button" example.  For that effect we need an extra div with the "input-append" class.  It isn't documented, but poking around in the simple_form code you'll find the :append wrapper which looks to do what we need.  Note that the lack of space between the input_field and the button_tag is important, otherwise you get a bit of whitespace in the resulting page.

  <%= f.input :name, :wrapper => :append do %>
    <%= f.input_field :name %><%= button_tag content_tag(:i, nil, class: 'icon-remove'), type: 'button', class: 'btn btn-danger' %>
  <% end %>

The result looks like:

Rails.root is the new Rails.configuration.root_path

In older versions of Rails the Rails.configuration.root_path method would give you the base directory of your application in the filesystem.  In Rails 3 this is now Rails.root.

Rails, as_json, undefined method `serializable_hash'

as_json was failing in one of my Rails models with:

NoMethodError: undefined method `serializable_hash' for ["id", 3]:Array

Eventually figured out that the problem was that I had specified a non-existent association to :include in as_json:


  def as_json(options={})
    super(:include => [:non_existent_association])
  end

Versioning RESTful APIs

There seems to be general (but definitely not complete) consensus among REST experts that the proper way to version a REST API is via custom MIME/media types.  You can Google this for yourself to find the discussions on the rest-discuss mailing list, Stack Overflow, various blogs, etc.

There was one subtlety to the discussion that I found interesting and I spent some time researching.  Most folks propose media types like:

"application/vnd.mycompany.myapp-v2+xml"

However a few folks, based on examples in the section of the HTTP spec for the Accept header, suggest a format like:

"application/vnd.mycompany.myapp+xml; level=2"

That's often countered with the argument that an "accept-extension" like "level" is optional (with the implication that it can be ignored if present, which seems like an unfounded assumption), or that it should only be used to indicate minor versions.

However my reading of the examples in the HTTP spec and related historical documents leads me to believe that using "level" to indicate major versions was exactly the intended purpose. All usage of the "level" extension historically seems to have been in relation to the “text/html” MIME type, and specifically to indicate major versions of the HTML spec (HTML 2, HTML 3, etc.) This page (http://www.w3.org/MarkUp/table-deployment) dated 1995 clearly shows usage of a "text/html; level=3" media type to indicate to HTML 2 only clients that they won’t understand the markup and should not attempt to do so.

That said, this is largely an academic exercise. In the absence of specific definition it seems like media types must be treated as opaque strings, so "application/vnd.mycompany.myapp-v2+xml" and "application/vnd.mycompany.myapp+xml; level=2" both serve the same purpose. Since there seems to be consensus that the former is the better format to use I’ll be going that way, I just don’t see any reason that the later format is any less valid. And in fact using the "level" extension for minor versioning seems counter to historical precedent, but (IMHO) acceptable and harmless if used on a custom MIME time.

Ruby, OptionParser, and Switches with Optional Arguments

Ruby's OptionParser supports switches with optional arguments. Both the official documentation and the pickaxe book give examples of various syntax choices like these:

"--switch [OPTIONAL]"
"--switch [=OPTIONAL]"
"--switch" "=[OPTIONAL]"

Somewhere along the way the first time I needed a switch with an optional argument I copied the third syntax, where the argument is specified as a separate string.

Later I noticed odd behavior. While my switches with required arguments could be used with or without an equals sign, my switches with optional arguments required an equals sign. I.e. "--switch foo" did not work for the switches with optional arguments, while "--switch=foo" did work.

I've lived with this oddity for several years. But then recently I picked up a copy of the new "Build Awesome Command-Line Applications in Ruby" book and was giving it a read. In their section about OptionParser they only listed the first syntax (all one string, no equals sign). That got me wondering if the
equals sign made a difference.

I went back and re-read the OptionParser section of the pickaxe book and noticed an interesting statement differentiating "--switch=ARG" from "--switch ARG". Specifically they say the later syntax "specifies the argument follows the switch."

Hmm, that sounds promising. Write a quick test. Yes! "--switch [ARG]" makes the argument optional, and with or without the equals sign. I should have stopped there, but now I was curious. So I tried "--switch=[ARG]", and oddly it works the same way. Optional argument, optional equals sign. Hmm, that's odd. Try the "--switch" "=[ARG]" syntax again just to make sure I'm not going crazy. Nope, not crazy. Argument is optional, but equals sign is not.

So, a bit more pondering and I come up with 9 possible syntaxes and set about to try them all:

"--switch [OPTIONAL]"
"--switch [=OPTIONAL]"
"--switch =[OPTIONAL]"
"--switch" "=[OPTIONAL]"
"--switch" "[=OPTIONAL]"
"--switch" "[OPTIONAL]"
"--switch[OPTIONAL]"
"--switch[=OPTIONAL]"
"--switch=[OPTIONAL]"

Interesting, all 9 are valid syntax for OptionParser. 8 work in terms of the argument being optional, "--switch =[OPTIONAL]" being the odd one where the argument is required despite the square brackets (tested with both ruby 1.8.7 and 1.9.2).

And finally, drumroll please, the distinction that actually makes the equals sign required is when you specify the argument as a separate string. The presence or position of the equals sign when specifying the switch to OptionParser makes no difference, except as noted with the "--switch =[OPTIONAL]" syntax. "--switch [=OPTIONAL]"? No equals sign required. "--switch" "[OPTIONAL]"? Equals sign required.  Wacky.

+1 to the Pickaxe book for prompting me to ponder the different syntaxes, but -1 for being somewhere between misleading and wrong in actually documenting the difference.

FIOS Router CLI and DHCP


I recently got Verizon FIOS service.  The router Verizon provides is an Actiontec MI424WR.  Verizon goes to some trouble to hide the Actiontec branding; but, for example, the Model Name field in the System Monitoring tab on mine says "MI424WR-GEN2".

Anyway, I wanted to get the DHCP lease database out of the router in order to populate that information into my own DNS server.  I initially looked at scraping the web interface, but the router does some weird Javascript hashing of the password before submitting it to the server.  I figured there had to be a better way.

In the advanced options you can turn on telnet access to the router.  That gets you into a command line administration utility, which in turn will let you access a shell (via "system shell").  However I was unable to find a DHCP lease database in the filesystem.  There was a /var/state/dhcp directory, but it was empty.

Some searching on the web turned up the fact that the Actiontec MI424WR runs a commercial Linux distro called OpenRG.  More searching turned up an OpenRG Configuration Entries Guide.  A bit more poking and I found that this command will dump out the DHCP lease database:

conf print dev/br0/dhcps/lease

Python Packages

I'm new to the Python and Django world, having primarily worked with Ruby and Ruby on Rails for the last few years.  In reading about how to manage dependencies several folks referred to pip and distribute as the new hotness (literally).  Ok, fine, they're probably something like ruby gems, but what's the difference?

Looking over their respective pages (http://pypi.python.org/pypi/pip and http://pypi.python.org/pypi/distribute) it is confusing to see that there appears to be some sort of circular dependency.  The pip page says "In order to use pip, you must first install setuptools or distribute."  The distribute page gives instructions for installing distribute via pip.  Wha?!

The heading at the top of the pip page says "pip installs packages. Python packages."  The heading of the distribute page says "Easily download, build, install, upgrade, and uninstall Python packages".  Argh, still confused.

Pip is more clearly documented, and serves in the roll of package downloader, dependency resolver and installer.  This would be analogous to yum or apt.  I.e. pip does not create packages.  Pip apparently replaces something called easy_install.

So if you're just using existing packages you'll just use pip, possibly in combination with virtualenv for isolation (similar to bundler in the Ruby world).

So what about distribute?  The distribute documentation is terribly confusing.  It says it replaces setuptools, but then seems to imply that it is setuptools.  It looks like distribute is a fork of setuptools because the setuptools maintainer stopped maintenance.  And I guess in parallel there's another tool called distutils.  These all are used to make packages.  As is usual for package making there's a metadata file (setup.py in this case) and setuptools/distribute/distutils take that and a directory full of files and makes a package.

And apparently because setuptools, distribute and distutils all have problems there's work underway on distutils2 to replace them all.

On a side note, you'll also find references to eggs.  This is apparently a packaging format introduced by setuptools.  It seems to have lost favor.  The PyPI repository distributes packages as generic tarballs.

So, in summary if you're making a package use distribute or distutils, and if you're installing a package use pip.  I think.  :)

Other references:
http://guide.python-distribute.org/
http://packages.python.org/distribute/

Upgraded to Mac OS 10.7 (Lion)?  Tried to update Macports with sudo port selfupdate and get:

Error: /opt/local/bin/port: port selfupdate failed: Error installing new MacPorts base: shell command failed (see log for details)


You probably still have the Snow Leopard version of Xcode installed.  Try to run Xcode, it will probably tell you that you need to upgrade.  Go delete the Install Xcode app from your Applications folder, fire up the App Store app and get Xcode again.  That will get you the installer for Lion, which you can then run to upgrade Xcode to the Lion compatible version.  Then the Macport update should work.

I'm working on a MacRuby project in Xcode.  I upgraded Xcode from 4.0 to 4.0.1 and Interface Builder stopped working right for my project.  New outlets weren't showing up so I couldn't connect them to UI elements.  I eventually realized that in the Xcode upgrade the MacRuby templates got wiped out.  A reinstall of MacRuby got things working again.

Filesystem Hierarchy Standard for Mac OS X

I frequently refer to the Filesystem Hierarchy Standard when trying to figure out where to put files on Linux systems.

I recently started doing some Mac development and was looking for an equivalent document. Apple's File System Overview seems to cover similar ground.

User-Friendly RESTful Routes in Rails

RESTful routes created via map.resources use the :id field for URLs for individual resources.  I.e. if user 'joebob' has an id of 42 then the URL for his resource is http://example.com/users/42

I find those URLs rather user-unfriendly.  Wouldn't http://example.com/users/joebob be nicer?

If you plan for this from the start you can fake out rails by using a non-standard id column.  I.e.

    create_table :users, :id => false do |t|
      t.string :id, :null => false
    end
    add_index :users, :id, :unique => true

Judging by blog posts there are at least a few folks out there who have taken this approach.  Here are a couple that point out issues you might run into if your user names (or whatever identifier you use for id) contain special characters:  here and here

But what if you already have a project up and running and don't want to rearrange your database?  Or just think that faking out Rails that way is kinda ugly?

It is possible you can do this somehow with map.resources and its :requirements parameter, but I haven't been able to figure out a syntax that works.  It seems that map.resources really expects to map the identifier to the id column.

So for now I'm settling with a partial solution.  In routes.rb:

  map.connect 'users/:name', :controller => 'users', :action => 'show', :name => /[a-z]+[a-z0-9]*/
  map.resources :users

The trick with the connect route is that the regex for name must not match ids.  I.e. you want /users/joebob to use the connect route, but /users/42 to use the RESTful route.  Of course if your usernames are alpha only this is easier.  Note that Rails does not allow anchor characters in the regex, so a regex like /^[a-z][a-z0-9]*/ doesn't work.

OK, but if you have a standard RESTful controller the first line of the show method is going to look like:

    @user = User.find(params[:id]

But that method is going to get passed params[:name] for calls using the connect route.  So you'll need to replace that line with something like:

    @user = nil
    if params[:id]
      @user = User.find(params[:id])
    elsif params[:name]
      @user = User.find(:first, :conditions => ['name = ?', params[:name]])
    end

At this point at least the basics are working, but I'd welcome feedback and suggestions for improving this.

Update:  See http://railscasts.com/episodes/314-pretty-urls-with-friendlyid

I needed to set the UID for an account on my Mac OS X 10.6 (Snow Leopard) box. Various posts online said that you can right click on an account in System Preferences -> Accounts to access the Advanced Options. But right clicking wasn't doing anything for me. Finally I tried a Ctrl-click, the standard way to "right click" when you only have a one button mouse, and that worked. My right button works everywhere else, but for some reason in this instance it didn't.

If your Ruby app using https or some other form of SSL is complaining about:

net/http.rb:567: warning: can't set verify locations

Then you probably passed in a bogus path to your CA file or path. In my case I had:

http.ca_file = '/etc/pki/tls/certs/ca-bundle.crt'
http.verify_mode = OpenSSL::SSL::VERIFY_PEER

That path is right on Red Hat, but I was running the script on my Mac laptop. Whoops. Fixed the path to something valid and the error went away.

I've been using the workaround shown at http://blog.zenspider.com/2008/05/httpsssl-warning-cleanup.html for a while now to eliminate the "using default DH parameters" warning you get when using https in Ruby. However, I recently tried to reference two of my libraries which contained the same workaround in a script and ran into a "stack level too deep" error as the two sets of code both tried to alias the use_ssl= method. I spent some time working on this today and I think I've come up with a better method.

Net::HTTP.ssl_context_accessor(:tmp_dh_callback)
http = Net::HTTP.new('www.example.com', 443)
http.tmp_dh_callback = proc {
  OpenSSL::PKey::DH.new(IO.read('dhparams')) }
http.use_ssl = true

There are a few options for how you create the Diffie-Hellman params. The code above assumes you've got a file with pre-generated params (this is perfectly acceptable from a security standpoint). You can create the file with the "openssl dhparam" command, or a little Ruby script. You could also generate the DH params on the fly, although this can be quite slow:

Net::HTTP.ssl_context_accessor(:tmp_dh_callback)
http = Net::HTTP.new('www.example.com', 443)
http.tmp_dh_callback = proc { OpenSSL::PKey::DH.new(2048) }
http.use_ssl = true

Or if you just want the warning to go away and aren't concerned about the security implications you can just use as empty proc as zenspider did. I've tried to look at the Ruby OpenSSL library source and figure out what happens when the proc doesn't return anything but the code is a bit too obtuse for me to figure out.

Net::HTTP.ssl_context_accessor(:tmp_dh_callback)
http = Net::HTTP.new('www.example.com', 443)
http.tmp_dh_callback = proc {}
http.use_ssl = true

Excel on the Mac defaults to portrait mode. Nearly all spreadsheets I create I want in landscape and finally tracked down how to set this as the default.

• Open up a new document
• Set it to landscape via File -> Page Setup
• Go to File -> Save As
• Switch to the /Applications/Microsoft Office 2008/Office/Startup/Excel folder
• Set the filename to Workbook
• Uncheck the Append file extension checkbox
• Save
• Quit and restart Excel

Your workbooks should now be landscape by default.

You can find this in the Excel help. Don't use the Help menu item on the main Mac menu bar, Excel's help isn't available there. Instead click the Help button on the toolbar in an Excel document to bring up the Excel help system and look for the "Control how workbooks and sheets are created" document, available under Contents -> Customizing Excel.

How to insert a new line above the current line in TextMate? A la capital O in vi? I couldn't find it in the TextMate book, nor searching around online. A few seconds of fiddling on the keyboard turned it up though: command-option-return

This makes sense, as command-return inserts a new line below the current line. Having now looked around some more this command-option-return key sequence is documented as "move to end of line and insert end-of-line symbol", e.g. a semicolon in C or Perl. Conveniently, in Ruby and presumably other languages with no end-of-line symbol it has been redefined to the "new line above current line behavior". It's disappointing that there isn't a universal shortcut.

Update: SourceForge no longer offers the Wikispaces wiki, this post is left up for historical curiosity.

The default wiki for SourceForge projects is used to be called Wikispaces. The CSS stylesheet configuration on the SourceForge Wikispaces is such that list items (HTML "li") are smaller than normal, and shrink as they are nested. Top level list items are smaller than I would care for, and by three or four levels of nesting they are unreadable. This has bugged me for a while, but today I was trying to create a page which had up to five or so levels of nesting (to represent a directory tree) and decided to see if I could fix this.

Using the wonderful Web Developer extension for Firefox I was able to pinpoint the source of this shrinkage, this line of http://static.sourceforge.net/css/sfx.php?secure=0&20080417-1657:

#frame li, #fadbtm li { font-size: 82.3%; }

Wikispaces uses about five CSS stylesheets including sfx.php, and they allow you to edit one of the other CSS stylesheets under Manage Space -> Look and Feel -> Edit your wiki stylesheet.

When editing that stylesheet Wikispaces restricts you to applying styles to the "wiki" class. Your wiki content is placed in an HTML div of class "wiki" and id "content_view". So I tried to add a line to the stylesheet which sets the font size for list items in class "wiki":

.wiki li { font-size: 100% }

However, there is a higher level div with id "frame" which the Wikispaces-provided CSS line applies to. Per the CSS specification styles applied to a specific id (#frame) take precedence over styles applied to a class (.wiki). As such the style for the id "frame" overrides my style for the "wiki" class.

Another read through the CSS spec reveals the important flag to override the normal priorities. And that did the trick, adding the following line eliminates the shrinking list items:

.wiki li { font-size: 100% ! important }

With the Oracle client 10.2.0.1.0 installer if you are automating the install with a response file, set UNIX_GROUP_NAME to something like say 'dba', then run the install but you are not in the dba group the install will fail with:

SEVERE:S_OWNER_SYSTEM_EPERM

I found nothing useful via a search for that error so I thought I'd post something. It would seem that you need to set UNIX_GROUP_NAME to a group you are in. Oddly none of the files that are written out end up owned by that group, so it doesn't seem like the installer really needs those privileges.

I'm trying to package the Oracle client for our internal packaging system, I'm not a DBA and wouldn't normally be a member of the dba group.