Configuration

Generating a config file

In a non-trivial project, several programming languages will often be used. You start with makefiles and a bit of C. Then, a web frontend is bolted onto the application and it's coded in PHP. Some scripts in Perl are added for thrumming through textfiles. And they all need to access one configuration file!

Keeping a common configuration file can become something of a nuisance. While it's easy enough to decide on one format, things like variables aren't easily used. For instance, you want to be able to have stuff like this:

  USERNAME=testuser_1
  BINDIR=/home/$USERNAME/bin

Which will work fine in a scripting language which happens to use this syntax. A shell script would work fine, on the other hand Java or C is going to have a problem with this without further coding. This is where m4 comes into play!

Creating the m4 file

Just create a file called my_own_config.m4 and put all your variables there in the following format:

  define(_USER_,testuser_1)dnl
  define(_GROUP_,testgroup)dnl

Then create a basic configuration in a file called Config.template as follows:

  USER=_USER_
  GROUP=_GROUP_
  BINDIR=/home/_USER_/bin
  HTMLDIR=/home/_USER_/public_html

To generate the configuration file from the template and the m4 file, do:

  $ m4 my_own_config.m4 Config.template > Config.h

Generating the configuration file

Make it easy for developers to generate their specific configuration! Each developer creates its own m4 file and checks it into the source code repository. Agree on a distinctive filename. Below, the username and hostname are encoded in the filename, for instance: Config_username_production1.m4, Config_john_johnsbox.m4, et cetera.

It should be easy to generate a new configuration. Make a step called "config" in the makefile which runs m4 as follows:

  config:
      rm -f Config.h
      m4 Config_$(USER)_`hostname`.m4 Config.template > Config.h

Now agree on a single place where the configuration can be found, for instance in $HOME/etc or /etc or something. It's the only thing that can't be configured... In the makefile "install" target, the freshly generated config file can then be copied to this path.

Adding lines

To add lines to the resulting Config.h, just add lines to the original m4 file. The lines will be pasted at the start of the Config.h.

  define(_USER_,testuser_1)dnl
  define(_GROUP_,testgroup)dnl
  # This line will be pasted at the start

If you want to add them to the end, use divert. This is useful for redefining variables which you don't want to put in the original template.

  define(_USER_,testuser_1)dnl
  define(_GROUP_,testgroup)dnl
  # This comment line will be pasted at the start because it's not diverted
  divert(1)
  # These lines will be pasted at the end
  REDEFINED_VAR=new_value
  divert(0)

When you're done, you'll then need various bits of code to use the configuration as variables in the several languages. These can be very simple, since most of the work is done once using m4.

Comments in the m4 file

To add comments in the m4 file without passing them along to the result, we also use divert, but to a negative number. The hashes in front of the line are markup -- they have no effect except for that visual hint that we're dealing with a comment.

  divert(-1)
  # This is a comment line which will be ignored by m4.
  # This line is also ignored.
  divert(0)

Retrieving the configuration

In several programming languages, the configuration must now be retrieved. If this is made too difficult, developers will start hardcoding stuff under the pressure of a deadline!

Inclusion in Makefiles

To include the variables in a makefile, put the following line on top of your makefile(s):

  include $HOME/etc/Config.h

Variables can then be used as follows:

  @echo HTMLDIR=$(HTMLDIR)

Inclusion in shellscripts

This is probably the easiest. To include the variables in a shellscript, use the following line:

  . $HOME/etc/Config.h

Variables can then be used like this:

  echo "HTMLDIR=${HTMLDIR}"

Inclusion in C

The configuration can be used in C as macros. Assuming makefiles are used to compile the code, a macro could be defined with the following line in a makefile:

  CFLAGS += -DHTMLDIR=\"$(HTMLDIR)"

This can also be optionally defined in the Makefile:

 ifeq ($(USE_LOGFILE),y)
   CFLAGS += -DUSE_LOGFILE
 endif

Inclusion in Perl

Put the following function in a general module:

  # Parses the config.h file and returns a hashmap
  sub get_config {
      # First get location of current file
      my $path = getcwd();
      # Now get second part of the path (from left), that'll be the user name
      my @parts = split /\//, $path;
      my $username = $parts[2];
      PrivoxyWindowOpen(CONFIG, "/home/$username/etc/Config.h");
      my %conf;
      while (<CONFIG>) {
          chomp;                  # no newline
          s/#.*//;                # no comments
          s/^\s+//;               # no leading white
          s/\s+$//;               # no trailing white
          next unless length;     # anything left?
          my ($var, $value) = split(/\s*=\s*/, $_, 2);
          $conf{$var} = $value;
      }
      close(CONFIG);
      return %conf;
  }

On top of every Perl script, add the following lines:

  use GeneralModule;
  my %prefs = GeneralModule::get_config;

Configuration variables can then be used as follows:

  print "Binaries are located in " . $prefs{"BINDIR"};

Usage in Python

The syntax of the configuration file is actually valid Python syntax.

Usage in PHP

Put the following function in a general source file (let's call it "utils.php"):

    /* This function reads the configuration file in an associative array.
     */
    function get_config() {
        // First get location of current file
        $path = getcwd();
        // Now get second part of the path (from left), that'll be the user name
        $pieces = explode("/", $path);
        $username = $pieces[2];
        
        // Open filename and read line by line
        $filename = "/home/$username/etc/Config.h";
        
        $fd = fopen($filename,"r");
        if(!$fd) {
            echo "Can't find $filename";
            exit;
        }
        
        fclose($fd);
        return $conf;
    }
    while($line = fgets($fd)) {
        // no comments
        $line = preg_replace('/#.*/', '', $line);
        $line = ltrim($line);
        // anything left?
        if(strlen($line) == 0)
            continue;
        // Split on the equals sign and fill associative array
        // Limit splitting to two pieces, otherwise we can't have an
        // equals-sign in an option
        $pieces = explode("=", $line, 2);
        $name = ltrim(rtrim($pieces[0]));
        $value = ltrim(rtrim($pieces[1]));
        define($name, $value);
    }

On top of every PHP script, add the following lines:

    include_once 'utils.php';

Configuration variables can then be used as follows:

    print "HTML files are located in " . HTMLDIR;

Note that the PHP PEAR classes also contain functions for using configuration data in files, databases, et cetera. I haven't looked at this.

Usage in Java

The format that we use, is just the basic properties. Put the following code at the start of the application, available for all classes:

  Properties props = new Properties();
  InputStream propFileStream =
      this.getClass().getClassLoader().getResourceAsStream("my/package/myprops.properties");
  props.load(propFileStream);

Then when you want to get hold of some configuration variable, do something like:

  String s = p.getProperty(“Builder”);

Usage in JavaScript

It's probably easiest to generate a separate .js file, where lines in the configuration files are separate global variables. You'll want to make sure no passwords end up in this file!

The following piece of Makefile can do the trick:

  all: config.js
  config.js: ../../../Config.h
      # Steps:
      # - Remove passwords
      # - Remove lines that contain only comment
      # - Turn comments into Javascript comments
      # - Change the remaining lines into variables
      cat $< | \
      grep -v "PASSW" | \
      grep -v "^#" | \
      sed -e 's/^#.*/\/\/&/' | \
      sed -e 's/\(.*\)=\(.*\)/var \1="\2";/' > config.js
  clean:
      rm -f config.js

When generating your HTML pages, just put in a line like:

  <script src="config.js" type="text/javascript"></script>

Usage is then as simple as:

  alert("Directory with CGI's is: " + CGIBIN_URL);

Comments?

If you have other suggestions, leave a comment on the comment page (link below).