I’ll start with a disclaimer: I know very, very little about file system design or implementation.

Nevertheless, I have been pondering it some lately and have some ideas of how I might like to see a file system implemented. So, this is arm-chair designing, and should not be taken as an informed discussion of the subject (however, I’d enjoy comments, including informed ones). In particular, I want to talk about what I’m off-the-cuff terming a “Convenience Operations Aware File System”.

In this model, the FS is aware of things like compression, encryption and versioning (i.e. “convenience operations” on a node that do not affect the contents of the node). When a node is saved, the following things could happen, in this suggested order:

1. Versioning - an rdiff is done on the node. In database terms, the current node is saved in the primary table with full content. The primary table has a has many relationship with a versions table that saves rdiffs from subsequent versions. Meta-data could indicate that only manual deletes of previous versions could take place or that they could be deleted based on other circumstances (space, time, etc).
2. Compression - A library or system call returns the compressed content. Ideally, this call could take as arguments the specific compression type and any options. The compression settings would be part of the meta-data for the node; rather than a zip file, there would be a node that is zip compressed. If the node is versioned, the content would be compressed, and also the rdiffs (probably all together) would be compressed. Hence the data itself would be rdiff’ed rather than the compressed data. If the compression information is changed, the versioned files will be re-/de-compressed as needed.
3. Encryption - Same idea as compression.

Possibly, folders would have the potential to be treated as a single node that contains a child file system, so that a folder could be subject to the above, for operations such as tar.

File operations, such as copy/pastes could retain these “convenience operations” or not. When copying to another file system, they could convert into a more conventional format, say a zipped file, without version.

So, there’s that thought.

First of all, I don’t believe in the “good shared web hosting provider”. They’re probably out there, and because they’re good, they’re probably expensive. But at that point, if you know what you’re doing, why not use a self-managed leased server? And if you don’t, it’s probably best to give the whole kit-and-kaboodle of the website over to a company that knows what they’re doing. So, it’s not surprising that “good, solid service” is not a priority for most shared hosting companies–it doesn’t make economic sense. Now, I’m not saying that to justify iPowerWeb, but merely to say it’s more funny than surprising.

When selecting a host for one of my earliest website clients (before I knew enough to use self-managed servers), I chose iPowerWeb. The service was fine, price was right. As far as the actual hosting experience, I was satisfied. However, I switched the site over to my server a couple of years ago when I switched from PHP to Rails for the storefront of this site. I called iPowerWeb and canceled the shared hosting account, explaining we just didn’t need it anymore. That’s where the fun starts.

A few months later, it came time for renewal, and iPowerWeb automatically renewed the hosting. Now, I’ll admit I should have actually read the “We’re about to renew your account” email and called then. But, I ignored it, figuring it was just a reminder email that hadn’t been turned off. Of course, they did charge. I didn’t have any documentation on the first cancel, so I called them again, canceled again, didn’t bother pushing the ”I’ve already canceled” too hard. Which I should have.

A year later, poof, another charge. Fed up, I had the credit card company do a charge back. Bearing all that in mind–that for about a year and a half, my primary goal with respect to iPowerWeb has been to cancel this account–I receive the following email this morning:

Dear Customer,

Your account has been suspended for a chargeback and please contact us @ 1-888-511-4678.

Thank you billing@ipowerweb.com

And I just had to laugh and share it with the world.

Oh, by the way, I use Layered Technologies for my two self-managed servers, and have been very pleased. I was particularly pleased that they upgraded the control panel to one with more services, such as remote reboot, and included existing customers in this upgrade. That tells me they’re not interested in just coasting.

Took a couple of weeks off of blogging for personal sanity. But I’m back now. Naturally, I have a half-dozen (I’m guessing here) partially developed ideas of what I might blog about this week. And I’m going to go with the one that most recently came to mind. Yeah!

By way of introduction: Back in Rails’ pre-1.0 era, I decided to develop a system for easy-to-create car dealership websites. The basic idea is that a bunch of sites would share the same code base with slight changes (primarily design) for each. Back in those days there was a productize plugin, but alas, by the time I was ready to use it, it was out-of-date as Rails rapidly developed in that time frame. I discovered Rails engines about that time, and was able to use them very successfully for this purpose.

Although I’m generally not doing sites for clients anymore, I think I’ve come up with another way to productize your rails app. This productize method fits a particular need, in which each product has different views but always shares the same models and controllers; more of theming, actually. This requires no plugins, so the “eek, a possibly business-logic-related plugin, let’s overreact” elements of the rails community can be happy. Also, I haven’t actually fully tried this, so it might not work.

Ready?

3. In config/environments/[product].rb, add: config.view_path = "products/\[product]"
4. In config/database.yml, add:

[product]:
  [database info for this product]

2. In config/mongrel-[product].yml (or equivalent changes for your web server setup). add or change:

  environment: [product]
  docroot: [rails root]/products/[product]/public

You can then place view code in products/[product], with public files in products/[product]/public. Then start mongrel with the mongrel-[product] config file. Then again, it might not work at all. I’m using some of these elements on a site I’m developing, but not all. I do think that at least the basic idea is sound.

Playing with config.load_paths would probably be a good next step.

I discovered Subversion and Revision control software in general a little over a year ago. I’d heard of it all before but that was when I began to learn about and use it. The appeal, once I took the time to think about, was significant. As a freelance website developer (and reluctant designer), being able to version my code meant time saved from searching through backups (when I’d deleted stuff) and cleaner space (from when I wouldn’t have deleted stuff; see previous point). It particularly allowed me to get rid of goofy versioning mechanisms I had employed (file names with 1,2, etc at the end, e.g.).

Having used Subversion extensively this past year (although certainly not all of its features), I have a few suggestions for newcomers regarding the repository setup. It turns out, looking back at it, that the online manual could have told me this just as well (more or less), particularly the section about Repository Deployment and Externals Definitions. If you’re seriously looking for repository layout advice, I recommend reading those more than the rest of this article. On the other hand, personal experience can at times be more immediate and useful that a manual. (By the way, I haven’t looked closely to see how well my suggestions really match up with the manual; rather, I’ve noted in re-reading sections that they make a lot more sense to me now.)

When I setup my repository, I set up one, single repository for all my projects. And life was good. The root directory contained a directory for each project. This was manageable up to about ten projects. Then, I begun dividing them into groups. My logic for a single repo was:

1. Aesthetic reasons: for better or worse, I just liked the idea of a single repository better.
2. Ability to svn copy files/directories between projects: A number of the websites I’ve made have very similar code that is not quite similar enough to just create a library, and the chance to save space appealed to me.
3. I could setup a frame project (for Rails apps) and when I started a new site, just svn copy).

I still aesthetically like the idea of a single repository, but that’s no justification for a business decision (at least when there are weightier factors). What made me change my mind otherwise? Simply, it became a headache. One of the problems was that the project group directories I created early on are no longer as useful groupings as they were, and I don’t really want the change to be versioned. When going back to earlier revisions, the path to the project would be uselessly different. But the straw that did in that design was one project. This project contains a lot of media. I didn’t want to put all that media in my main repository. I like the fact that it doesn’t take up a lot of space. But I’ve found a need to start versioning many of those large media files. So, I decided to use svndumpfilter to break out the project. And with all the path moves, I realized I was facing a serious chore. I also had a bunch of project folders that were basically empty, unused, I’d created in a fit of “planning”. There was no reason to have them at all, so I wanted to completely remove them from the repository.

Over the past couple of weeks, I’ve spent many hours writing a lengthy script to break up the repository. Because I realized it’s best to get it done now. Some major difficulties have been the result of my earlier reasons for a single repository. The copies between projects and setting up projects by copying from a frame create extra dumps that must be done. And for very little gain, it’s turned out.

Here’s how my repositories will now be:

From the view of svn-serve–at least, for now–the repository won’t look much different. But the project group folders will generally be regular folders with the filesystem, holding project folders, which will each be (generally) a single repository. There are a few exceptions, where the group is the root of the repository, holding multiple projects. But I plan for minimal, if any, path moving once they’re created (if needed, I plan to do it using dumps, but I will try to plan to avoid it).

Anyway, my five key points of advice:

1. Except for small, easily grouped projects (rails plugins, for example), each project has it’s own repository.
2. In any case, project heirarchy (divsion into groups) should not be within the repository. There’s no use in having that versioned and it gives much more freedom.
3. Use svn:externals rather than svn copy for sharing code between projects, even within your own repository realm. And when you do, use -r option to explictly set the revision (or set the external to a tag). Setting to the trunk without a -r is asking to break your code.
4. If you need a frame for creating new projects, make the frame itself a repository. Copy the repository (within the filesystem), rather than a project within a repository. Much more manageable. Or use a script.
5. It’s worth the extra space to copy large chunks from one project to another in different repositories, when a library shared via svn:externals won’t work. It’s better than the headache.

Yes, the proceeding blog entry was ~80% introduction. test

I’m using Liquid templates in a Rails project I’m working on (top-secret confidential don’t-ask). The important point is that someone else will be doing parts of the design, and there’s the potential for multiple themes, kept up by various people not me. I’ve used Liquid a good bit recently within Mephisto, but this is my first shot at incorporating into a project of my own. Although I have much to learn on using the templates, what I found particularly difficult was how to incorporate them as rails views. So, I thought I’d share from my current solution.

The actual Liquid call lie within a Theme model (some bits removed, I think irrelevant). render is the main method to consider:

require 'liquid'

class Theme
  # the path of the themes directory
  THEMES_PATH = RAILS_ENV == "test" ? File.join(RAILS_ROOT, "test", "fixtures", "themes") : File.join(RAILS_ROOT, "app", "views", "liquid")

  # Not necessary, but helpful
  def self.find(name)
    new name
  end 

  def initialize(name)
    path = File.join(THEMES_PATH, name.to_s) # The path to the particular theme
    if File.directory?(path)
      @path = path
      @name = name
    else
      raise LoadError, "Theme does not exist" # Ensure the theme actually exists
    end
  end
  
  # render is where Liquid is actually invoked.
  # This takes the +path+ to the template (relative to the theme root) and
  # +assigns+, a hash of variables that are made available to the liquid template.
  def render(template_path, assigns = {})
    template_path = File.join(@path, template_path + ".liquid") # The template, as in an action view
    layout_path = File.join(@path, "layout.liquid") # The layout template
    
    if File.file?(template_path)
      # Set up the liquid file system. Not really sure, here, but it works.
      Liquid::Template.file_system = Liquid::LocalFileSystem.new(File.dirname(template_path)) 

      # Liquid initializes and parses the template (action view)
      template = Liquid::Template.parse(File.new(template_path).read)

      # Render, taking the assigns arguments. Assigns will be available as liquid variables to the template.
      content = template.render(assigns)

      # Add the content as an assign to send to the layout
      assigns['content_for_layout'] = content

      # Repeat for the layout, which includes {{ content_for_layout }}
      Liquid::Template.file_system = Liquid::LocalFileSystem.new(File.dirname(layout_path))    
      layout = Liquid::Template.parse(File.new(layout_path).read)
      return layout.render(assigns)
    else
      return false
    end
  end
end

Then, I have a “lib/liquid_render.rb”

# Allows simple rendering of liquid themed templates

module LiquidRender 
  def current_theme
    # logic to get current theme
  end
  
  # +render_liquid+ can be passed url params or determine them from Rails' +params+ method.
  # It uses these params to determine the template path
  def render_liquid(url_params = {})
    url_params[:controller] ||= self.class.name.underscore[0..-12] # Not very pretty, but it works.
    url_params[:action]     ||= self.params[:action]

    # Call the +current_theme+'s +render+ method.
    text = current_theme.render("#{url_params[:controller]}/#{url_params[:action]}",
                                instance_to_assigns)
    
    # If render worked (i.e. template exists), call render :text
    if text
      render :text => text #instance_to_assigns.inspect # 
    else
      render url_params
    end
  end

  def instance_to_assigns
    # Returns assigns based on controller instance variables
  end
end

Then include LiquidRender in ApplicationController:

class ApplicationController
  include LiquidRender
end

and, now, you can (note the format.html line):

  def index
    respond_to do |format|
      format.html { render_liquid }
      format.xml  { render :xml => @profile.to_xml }
    end
  end

Okay, that may have not actually been that helpful. But maybe it is…