Hello WordPress!

In a probably not unusual manner, my website has evolved through a progression of technologies.  After working my way through plain HTML, PHP, and a PHP framework (Kohana), perhaps it is time to try some blogging software, in this case, WordPress.  I have written articles/posts using the other mechanisms, but this will allow for writing more text instead of code.  Let’s see how this experiment unfolds.

IPCop Configuration Backup

Contents

History

The intent of this enhancement was to implement a feature I felt was missing from IPCop. Though IPCop had a mechanism for backing up the configuration to a floppy disk, this would obviously not work if a floppy drive was not present in the machine. And, if a floppy drive was present, it could be inconvenient to use, especially if the floppy disk needed to be left in the machine or if the IPCop machine was not easily accessible. So, I set out to remedy that.

My solution was to allow creating a configuration backup archive through the web interface that could then be downloaded, uploaded, and restored. This would allow for configuration backups and restores to be done from any machine that could access the IPCop web interface. The first implementation was based on IPCop v1.2. It provided the basic functionality, but was not secure, and was not especially easy to install as that had to be done manually. The current implementation is based on IPCop v1.3.0, provides the missing security, and has an install script to make that part a little easier. IPCop v1.2 is also supported.
Note: This functionality is included in IPCop starting with v1.4.0. Because of this, the files that were here are no longer available for download. This page is left for documentation purposes.

Functionality

The basic feature set consists of:

  1. Backup configuration to an archive
  2. Restore configuration from an archive
  3. Download configuration archive
  4. Upload configuration archive

This provided the core functionality and the first implementation had this capability. To that, we need to add security, but still keep flexibility for setting up new IPCop machines and for managing multiple IPCop machines.

The basic requirement for security is:

  • Do not allow restoring an archive not created on that IPCop machine.

This requirement is met by encrypting/decrypting the archive using a random key that is generated and stored in a file on that IPCop machine. The randomness helps ensure uniqueness among different IPCop machines. Storing the key in a file allows restoring archives without having to manage the specifics of each individual archive that is created. Encrypted configuration archives have a .dat file extension.

The flexiblity requirements are:

  • If it is a new IPCop setup, the configuration from another machine should be installable.
  • Make managing multiple IPCop machines easier in this implementation compared with the first implementation.

The determination as to whether it is a new setup is whether the generated encryption key file exists. Once a configuration archive has been created and an encryption key file has been generated, that machine is no longer considered new. Unencrypted configuration archives have a .tar.gz file extension. In an effort to make the downloading/uploading of configuration archives easier for multiple machines, the base of the configuration archive file name is the name of the machine. For example, if an IPCop machine is named ipcop123, the configuration archives will be named ipcop123.dat and ipcop123.tar.gz on that machine.

To briefly recap what gets created and what gets restored, it is this:

  1. Creating a configuration archive results in both encrypted and unencrypted archives.
  2. If an encryption key file does not exist (new IPCop setup), creating configuration archives will generate the file.
  3. An unencrypted archive (.tar.gz) can only be restored on a machine that does not have an encryption key file, i.e., an archive has not yet been created on that machine.
  4. An encrypted archive (.dat) can only be restored on the machine it was created on because a matching encryption key file is required to decrypt the archive.

Using The Backup Web Interface

A web interface is provided for using the configuration backup functionality. It is located by navigating to System and to backup from the menus. A box surrounds the backup information and controls. The information box at the bottom of the page displays the output performed during a backup to floppy disk. If an error occurs during the upload, creation, or restoration of an archive, a message will be displayed in an error box above the backup configuration box. Normally, the error box is not displayed.

The backup configuration box is divided into three sections. The top section identifies the configuration archive files. The names used for the archive files are displayed on separate rows for the encrypted and unencrypted archives. If the archive file does not exist, a message to that effect is displayed. If the file does exist, the date and time of the file is displayed along with an Export link for downloading the file.

The center section contains the controls for managing the archives. The first row has the Create and Restore buttons. The Create button creates new configuration archives on the IPCop machine. If it is the first archive creation on that machine, the encryption key file is also created. The Restore button restores the configuration from one of the archive files displayed in the top section of the box. After restoring an archive, a reboot is suggested.

Below the row of Create/Restore buttons are controls for importing/uploading an archive file. An entry field for a local file name is on the left side. Immediately to the right of the entry field is a Browse button for locating a file on your computer. On the right side is an Import button. The full name of the button indicates what type of file to import. It will say Import .tar.gz if an unencrypted .tar.gz archive file can be imported. It will say Import .dat if a matching, encrypted .dat archive is required.

The bottom section is for backing up to a floppy disk. The Backup to floppy button initiates the process. In order to back up to a floppy disk, the floppy disk must be in the drive before the button is pressed.

Implementation Details

In addition to the changes to the backup.cgi web page, there are two new binary files for backing up a configuration and restoring a configuration. These are named ipcopbkcfg and ipcoprscfg respectively. The source code for these is available separately. The backup web directory is protected with an httpd.conf entry. Additional languages entries are added to the en.pl file.

The encryption key file is generated by using ipsec ranbits 256. The encryption/decryption of the .dat archive is done using openssl des3.

ipcopbkcfg — If the encryption key file does not exist, one is created. The unencrypted .tar.gz archive is created and then encrypted to the .dat archive.

ipcoprscfg — If the encryption key file exists, the encrypted .dat must also exist. If it does, it is unencrypted to a temporary .tar.gz file. if the encryption key file does not exist, the encrypted .dat file must not exist either. The unencrypted .tar.gz is copied to a temporary .tar.gz file. A temporary directory is created for testing the .tar.gz file, which is untarred to that directory. The temporary directory is then removed and the real untar done to replace the configuration files.

install.sh — The install script handles untarring the archive, creating the backup directory, setting the file and directory, permissions, adding the ignore entry to exclude.system, including the backup.conf in httpd.conf and adding the new language entries to the en.pl language file. This makes for a much simpler installation process than the prior implementation.

Download Files

These files work with IPCop v1.2 through v1.3.0. If you have a problem with them, please let me know. Caveat: Only the English language text is included at this time. Note: This functionality is included in IPCop starting with v1.4.0. Because of this, the files that were here are no longer available for download. This page is left for documentation purposes.

  • ipcopbkcfg.tar.gz (New backup files)
  • install.sh (Installation script)
  • ipcopbkcfg_src.tar.gz (Source files for new binaries)

Installation Instructions

  1. Use SCP to put the files on the IPCop machine.SCP ipcopbkcfg.tar.gz to /tmp/ipcopbkcfg.tar.gzSCP install.sh to /tmp/install.sh
  2. SSH to the IPCop machine and log in as root
  3. Change to the /tmp directory# cd /tmp
  4. Make the install script executable# chmod +x install.sh
  5. Run the install script# ./install.sh

After the above, the new backup functionality should be installed and ready to use.

IPCop

IPCop is a very capable open source firewall. I highly recommend it. For more information, follow the title link. This page is to document my additions to it.

IPCop Configuration Backup

This enhancement is to increase the configuration backup capabilities of IPCop. IPCop v1.2 through v1.3.0 only allowed for backing up the configuration to a floppy disk. This enhancement allows for backing up the configuration through the web interface.

Note: This functionality is included in IPCop starting with v1.4.0.

Web Site Evolution, Part I

Or, From Static to PHP

Just a little background

I have had a web site of some sort since around 1995. What you see on this site is, of course, the current implementation. As with most sites, this is not how it started. Though I have worked with some of the technologies used to created dynamic web sites, I have not used them to my own web site until fairly recently. For a very long time, I had a purely static web site, static HTML pages and mostly static content, too. Between then and now, I have experimented with various approaches, still having a mostly static result.

Other than just adding some simple links pages, the early changes were to add some of my photography to the site just to add some spice. This helped some, adding some visual interest if not much else. For a time, I experimented with the ArsDigita Community System(ACS). This web application framework was written in Tcl, ran on AOLserver and used the Oracle database server. This had the capability of creating dynamic web sites, though it also included an associated cost that made it prohibitive for my to use long-term for my personal web site. OpenACS offered an alternative to this. Based on ACS, it used the PostgreSQL database instead of Oracle. Also written in Tcl and running on AOLserver, this provided the capability for dynamic web sites with a lower cost of entry. While fun to experiment with for a while, I felt the need for something more, more mainstream that is.

Part of the desire to use mainstream technologies was the desire to development marketable skills. So, during the ACS experimentation, I also looked at having a Java based web site. This testing and evaluation was done using the Apache web server combined with the Jakarta Tomcat Java Servlet container. This, too, was fun to work with and more popular than Tcl and AOLserver. Further experimentation used Perl and Python with the Apache web server, two other popular solutions. Eventually, I ended up with the framework I have now, which seems to be growing in popularity.

What next?

Developing web applications at work with Microsoft’s ASP demonstrated the ease of development offered by scripted web pages. The addition of VB COM components handled the intensive tasks. This combination allows for fairly rapid development, with compiled code providing increased speed where necessary. With my personal web site, I do not expect to have traffic that would require compiled code. It is possible some portions in the future may be compiled, but I expect that to be the exception more than the rule. As for the Microsoft tools, I am also reluctant to spend the money required to license the servers and development tools for creating and running a personal web site. That is not to say that I will not do that in the future, only that I have not done it for a while. During my independent consulting days as BSoft Productions, I did have a Microsoft MSDN subscription. When that was no longer tax deductible as a business expense, the subscription and the expense ended. Now that ASP.Net works with C#, it may be worth making the investment again.

But, I digress. The use of Microsoft technologies raises again the cost factor, which can be substantial. ACS was cost prohibitive due to Oracle and the skills to be gained using Tcl and AOLserver technologies used by OpenACS were not quite marketable enough. Web technologies have changed since then as they continue to do. The ease of development offered by scripted languages such as Perl and Tcl continue with new languages being added. One such language is PHP. After some experimentation with it, I decided to rework my web site using PHP as the basis. A web server running Linux, Apache with mod_PHP and using PostgreSQL for the database provides the capabilities for hosting a dynamic web site while still offering the low cost factor. With the growing popularity of PHP, the skills may have future marketability also, though possibly not as much as with Microsoft technologies. However, in order to pay now to earn later, you need to be able to pay now, which I cannot afford at the moment. The choice I made should be a nice compromise for personal use. While the syntax may be different, PHP web pages offer scripted classes and inheritance. I have heard that ASP.Net and C# offer this as well. Though I have not yet had a chance to work with the .Net technologies, I plan to do so when the opportunity presents itself, but I digress again.

Current implementation

Over a couple days time, I made some changes that will provide a framework for future enhancements. The first change was to add the Gallery software to allow for easily adding images and albums without requiring code changes. The second change was a bit more drastic, replacing the static HTML pages with a framework written using PHP. This framework is based primarily on the support of classes and inheritance offered by PHP. There is a base class for a Page, which the individual pages inherit from. Supporting the Page class is a Breadcrumb class and replaceable PageSection derived classes. The class inheriting from Page provides the actual content for the pages. The PageSection derived classes, PageTop, PageCenter and PageBottom, serve a supporting role to the Page classes. The use of CSS and replaceable pages and page sections derived from common base classes allow for easily changing the look of the entire site. The next article will have more details on the beginnings of the framework.

Proposed enhancements

The quick, two-day change is just the start of the work I have planned. While the use of a pre-written PHP framework may allow for a more rapid implementation, it would not provide the educational experience I also desire. Besides, I am not even sure such a framework exists and searching for it has not been a priority.

A few of the changes I have planned are to use the database for data storage. The first few changes are for the articles, links and bookshelves. I will provide more details on these in future articles, but here is a quick overview.

Articles: The current implementation for articles includes the page contents by reading the data from a static file. This was the quickest change from static HTML to using PHP, but does not provide the dynamic nature I wish for. The article data needs to be stored in a database according to topic. The first iteration will likely have the entire page data stored. Eventually, I would like to store different segments of the articles separately to allow for better formatting capabilities.

Links: Maybe I have too many. I definitely have too many that have gone stale. Moving them to the database will provide an easier way to manage this. A regularly running script can check the database for stale links, marking which ones have possibly gone stale or are just having the site reworked. After being stale for a time, they can be automatically hidden and marked for removal.

Bookshelf: I had started creating bookshelves to track some of the books I have on my shelves. As with the articles, I have books on many different topics. To present some of these on the web site, a database would provide an easier way to add new books without having to modify the pages to do so. I would like to have the ability to add new book entry simply by entering an ISBN number. The admin pages should be able to find the cover images to use for creating the links.

Conclusion

This seems like a lot of work for a personal site and it is given the time constraints of family life. While some individuals may benefit from the information presented, I feel I benefit the most. The educational experience alone is worth the effort. Besides, maybe the framework will become good enough to share or, at a minimum, others can learn from my mistakes.

Web Publishing

This is the part of my software development career I like best. The work is fun to do and, more imporantly, the result is much more visible than the applications used internally by a single department of some company. Those too are important. I just prefer having a broader audience to my work.

Until I read Philip and Alex’s Guide to Web Publishing, I used to think I knew quite a bit about developing web sites. This book has given me new insight into what makes a web site effective, contrary to the advise given in some of the previous books I have read. To get an idea on how much I have enjoyed this book, consider that there are very few technical books I have read cover to cover. Of them, this is the only one I have read twice. If you are at all interested in doing web development, read this book.

For a hardcopy, order from Amazon.com

What do I want to do with the web? I like the idea Philip espoused in his book, that web sites can be reliable, have good performance and still be relatively inexpensive to develop. This runs contrary to my corporate experience which typically allowed less than desired performance and less than desired reliability. Perhaps this is due to the platform being used. My corporate experience used the Microsoft Windows NT operating system intead of Unix/Linux. Another possible problem is having to relearn the problems inherent in the combinations of each new software release. For example, using Microsoft Site Server with Microsoft SQL Server 7.0 will have a different set of problems than using Site Server with SQL Server 6.5. It is true that there will always be some teething problems caused by combining different packages, especially when they come from different vendors. Having these problems between products from the same vendor is an embarrassment.

To learn about different solutions, I experiment on my home system, which is running RedHat Linux 6.2. Some of the technologies and tools being expermented with are:

Apache 1.3.12 is the web server running behind the Java application server. According to the Netcraft survey, Apache is the most popular web server being used.

Jakarta Tomcat 3.1 Part of the Apache Jarkarta project, Tomcat is the reference implementation for the Java Servlet 2.2 and JavaServer Pages 1.1 Specifications.

AOLserver – This is America Online’s open source web server. Before AOL bought the NaviSoft company, it went by the name NaviServer. The Netcraft survey recognizes this server by a combined name like “NaviServer/2.0 AOLserver/2.3.2”. Given the name, it should come as no surprise that this is the web server software running behind all of the AOL member sites. It is also running behind the Digital City web sites. One of my favorite features of this web server is the direct connection it has to the database, providing the capability for enhanced data access performance. Built in scripting using Tcl allows for a dynamic web site. Having it packaged as open source is an added bonus.

Oracle – This is the database used by the ArsDigita Community System. At my employer, Geneer, we have used both the Solaris and Windows NT versions. The version I am installing at home is 8.1.5, otherwise known as Oracle 8i.

ArsDigita Community System – ArsDigita, the company Philip Greenspun founded, has developed this free toolkit to ease web site development. Built on AOLserver and combined with a robust database and a stable operating system platform, it makes for an effecient combination of packages, one which has proven itself to work reliably.