BackEmUp is a simple, non graphical user interface incremental file backup program. It is able to backup an entire drive, or only certain subdirectories, or backup up everything except a user specified list of files/directories or specification mask. Hidden and System files as well as Extended Attributes are backed up. It can maintain multiple versions (generations) of a file. It can back up to any random access device: floppy disks, zip drive, another hard file, or a lan drive. Presently, it can not back up to sequential tape devices. It can use CD-R and CD-RW media for certain features. It has been tested on OS/2 Warp 4, Warp Server for e-Business, and Windows 95/98. Files backed up under one operating system may be restored to another. It uses a simple plain text ASCII control file that is easily viewable or editable with standard text editors (be sure to save it as plain text if you do edit it ... do not save it as a Rich Text or MS Word document!)
User exits can be optionally utilized to compress (zip) the data files as they are being backed up, or to encipher the backup for security.
Each file that is to be backed up must be able to fit onto one backup media disk. Enabling the compress option may effectively increase the amount of data that can be backed up to each removeable media disk.
Mixed backup media types (such as R/W disks combined with CD-R) can be employed using the concept of inactive volumes.
There is no expressed warranty with this product. You, the user, assume all risk and responsibility for any damage or data loss to your system. I will not be liable for anything as the result of your use of this product. This product may not perform as claimed in this document. If you do not agree that the author has no liability, nor that no guarantee is implied with this product, you are hereby forbidden from attempting to use it on any machine in any method, and must immediately delete it from your system. Use is strictly "as-is" and you assume all risks.
This program is offered as "Shareware" and may contain substantial defects.
Also, as it is command line driven, it is easy to incorporate into other scheduling utility scripts that automatically run at certain times of the day.
The esc key may not function under Windows due to the way Windows handles the keyboard. Also, under Windows, use the Control+Break key instead of the Control+C key. If the compression or cipher user exits are being currently executed, Windows may not allow BackEmUp to recognize the Control+Break sequence, so try repeatedly.
This is a short summary of some of the command line options. Entering BackEmUp with no parameters will give you the compete list of valid command line options.
Used for first time initialization of the control file. This will cause the creation of the control file, but no files will be backed up. Thus, the operator can edit the control file to apply special Exclude commands, or tweak other options.
This allows you to pass a dynamic cipher key to the cipher program. It will be substituted in the template(s) at the location specified by "$key".
Allows a way to uniquely identify individual backup sets such that additional protection is provided from mixing diskettes. For example, you may identify the backup set name as "my_docs" for the backup set containing your documents vs a backup set name of "drive_d" for data from all of drive d:. By default, the backup set name is "BACKEMUP".
This string is placed in the ID file on each media and provides the name of part of the ID file: i.e. BACKEMUP.ID or MY_DOCS.ID and also provides the name portion of the CTL file: i.e. BACKEMUP.CTL or MY_DOCS.CTL etc. Naturally, it should be 8 characters or less for manageability.
The user may issue a "type a:backemup.id" command to see the volume number, backup set name, and applicable source directory for that particular backup media.
You could use this BSNAME option as a "belt and suspenders" method of managing multiple different backups. Its up to you. For many people, having all the control files names "BACKEMUP.CTL" is perfectly fine. Others may want to use different names to help them remember what is what.
By default, when a backup volume is mounted it is automatically checked for consistency. Any file in the backup files subdirectory that follows the appropriate backup file name convention is verified to be referenced by the backemup.ctl control file. If a file exists but is not referenced, it is considered to be orphaned. Orphaned files will be listed (and kept), and by default, BackEmUp will wait for the user to acknowledge this situation. Optionally, this acknowledgement prompt can be skipped by specifying /orphans+ to instruct BackEmUp to effectively ignore these files. /orphans- instructs BackEmUp to automatically erase any orphaned files it finds.
A situation where orphaned files appear should not be considered normal.
Signals that a restore operation is to be performed.
Redirect the restore operation to a different location other than originally backed up from.
Restore generation nn of the file(s).
This has two flavors:
Combine this option with the /rpt option to further explore the effects.
Restore only the subdirectory information.
Causes a restore operation to rename the existing previously restored file
from its short name to its long name.
Note: This will only rename files. It will not copy files from the
backup media.
Typically, this option is only used when restoring a Windows system from a complete disaster where the initial critical files were restored under a DOS boot diskette.
This is short hand for a Report. When combined with the /rgen option, various information about what has been backed up will be output. /detailedrpt yields a more detailed report. The output is written to "stdout" and thus can be redirected to a file for printing or other purposes. You may also specify a file mask to report information about a particular file or set of files.
For example:
This option without a file mask specification will summarize the backup runs giving the backup run date and the number of files backed up at that time. If an "*" is displayed as the first character, it means that this generation has had some files roll off due to the generations specification.
Specifying a file mask (even it is "*" for all files) will display detail information about when each file was last backed up.
Specifying the /rpt option with a negative /rgen value will give detailed information about restoring to that particular date in time.
/list will scan for changes that should be backed up, but will not perform the backup itself. The output is a list of files which is written to "stdout" and thus can be redirected to a file for printing or other purposes. It will not verify the contents of the backup media for missing files.
/dtllist is similiar to /list plus listing the details of each file, including the reason it was selected for backup.
By default, the control file is "backemup.ctl" and it exists on the backup media. A copy also exists on the hard file in the logical root of the backup source. This is a copy for reference only. The program uses the copy from the backup media for all operations, thus there can be multiple backup sets from the same source specifying different options. Therefore, if you make any manual changes to the control file, edit the one from the backup media, and save it as a plain text file!
The control file consists of one line records. Each record is identified by a type as the first character. The following are record types:
This layout is subject to change. The fields in order are:
These are processed sequentially in the order listed in the control file. This can be used to exclude most of the files from a large application, but include the user customized files. For example:
This instructs the program to exclude all below Netcape except for the Users subdirectory and all below that.
If the Include/Exclude path does not specify a path (for example, "*.cfg"), the it will apply to any and all subdirectories.
The number of subdirectory levels to include. Default is 32767 which effectively means all subdirectories.
TRUE will cause the program to manage the "A - archive" attribute. Setting this option on will cause files that have the "A" attribute on to be backed up and the "A" attribute reset on the file. If this option is off, the program uses the file's timestamp and/or size to determine if the file should be backed up.
The number of versions (generations) of a file maintained on the backup set. Thus you can keep the most recent version and previous version(s) for restoration.
Operationally, the program will insure it has successfully created a new generation on the backup set before allowing eligible previous generations to be discarded.
This can be used to ensure there a adequate generations of a file on the backup media. Normally, a file is only copied to the backup media when it has been changed. This can override that by ensuring that there are nn copies of the file on the backup media even if it hasn't changed. Used with the distinct volumes option, you can be safeguarded against losing the only backup copy of a file if a backup volume becomes bad. For example, application program files may never change whereas the data files typically do. This option could be used to keep an extra generation (copy) of the program files on the backup media. Typically, you wouldn't need to do that as you would simply re-install your application from its installation media, however, this option provides just a little more flexibility. Obviously, this option must equal or be greater than the generations option.
This affects what happens to files that are excluded ( @ Exclude directive ) and not what happens to files that are normally erased from your source. Normally, if a file was backed up and is now erased from your hard file, the next backup will start removing the oldest generation from the backup set. Thus, the file will eventually be purged from your backup set. (It isn't done all at once just in case you erased something that really didn't want erase, in which case if you had multiple generations backed up, you might have a chance of getting if off the backup set before it is completely purged.) This flag has no effect on that operation. However, if your exclude mask specify that this file isn't going to participate in the backup process any more, this is what this KEEP_EXCLUDED flag controls.
If you Exclude a file (or bunch of files), enabling this option will prevent those files from being discarded from the backup set. If the option is disabled, then those files will start to be discarded as described above. Why would you want to do something like this? For example, you might want to back up some of last years data and then erase it from your hard file, but you want to be able to retrieve it later from your backup set. This option allows you to do just that by excluding those files explicitly and setting this option. Remember, if you don't exclude them they will start to be discard from the backup set. Also remember if you decide at some point to disable this option, then the purging will resume for all files that aren't existing anymore.
This option modifies the behavior when a candidate file no longer exists but does have a generation on an inactive volume. The default option is FALSE which will indefinitely maintain the record of the deleted file in the BackEmUp.ctl control file. Thus, if you do a restore operation, these deleted files will reappear as they are restored from the inactive volume. If the purge option is enabled (set to TRUE), then although the file will NOT be erased from the inactive volume, BackEmUp will discard the record of the oldest generation, thus it will be eventually purged from your backup set as described above for the Keep Excluded option. When you created your Read Only inactive volume, you should have copied the current BackEmUp.ctl control file to that media, thus for historical purposes, the file could easily be restored. By enabling this option, you will not get previously deleted files restored unexpectedly.
This option has no effect on the number of extra generations tracked on the inactive volume(s), thus if the generations option is set to 2, you could have 4 or more generations of an active file tracked: 2 generations on active volumes, and possibly 2 or more on various inactive volume(s). The thought process behind this is that if the file exists, then all generations on inactive volumes will be tracked to facilitate the greatest flexibility in restoring. If the file no longer exists, then records of its previous generations will be purged or kept depending on this Purge option setting.
This signals whether or not new media can be mounted in the backup device to extend the backup set to multiple media. This would be set to FALSE when backing up to a lan drive or another hard drive subdirectory.
This prevents multiple generations of a file from residing on a single backup media. Thus, for the "belt and suspenders" type of person, this will ensure that if an individual backup media becomes destroyed, that another generation may exist on another disk. Obviously, this is only applicable to removeable media, and if requested generations to keep are greater than 1.
This option applies to non removeable media, and is intended to limit the backup volume size on large drives such that the backup could be mastered onto a CD. It defaults to approximately 640 meg to allow room for other files and support programs to mastered onto the CD along with the backup volume information. This size can be edited in the control file, or by a command line switch, for example /MAXVOL713031680 would allow approximately 680 meg of backup files in the FILES subdirectory. Remember, each megabyte is actually 1048576 bytes and not a million! When specifying this option, do not use commas for readibility. Setting this option to 0 will disable this check, thus the size will only be limited by the physical size of the media.
These enable or disable the user exits to appropriate user supplied program.
These specify the system commands an options for these user exits. BackEmUp will substitute strings for $archive, $file, $dir during operation as appropriate. BackEmUp has provisions for working around long name restrictions with a compression program. i.e. For the restore operation, BackEmUp will use temporary files such that all intermediary operations use short file names and then put the final result into where it belongs with the appropriate long file name (if the operating system supports it). See the section on Disaster Recovery for more information.
If the corresponding option is enabled, edit these templates to match your specific user supplied program parameters.
Explanation of the sample templates provided
There are native versions of BackEmUp for OS/2, Windows, and DOS. The only significant difference between them is in long file name processing and Extended attributes. As hardly anybody uses DOS anymore, and DOS has severe memory and resource limitations, BackEmUp for DOS has been optimized for use during the Windows disaster recovery steps. Its control file read operation has been pruned to only consider files in the root and certain Operating system directories, such as you need to restore a minimal Windows system from the Windows Emergency Startup diskette. If for some reason you desire full support, use the /FULLDOS option.
OS/2 can handle long file names even when booted from its disaster recovery floppy disks. Therefore, its compress/uncompress templates include quotes around the portions of the file names that could contain embedded blanks. This would only be a consideration if you are restoring compressed files to a different operating system, and it is simply managed by putting the correct compress/uncompress command templates into the control file.
OS/2 uses extended attributes whereas DOS and Windows do not have the concept. They will be lost if you backup on OS/2 and restore to Windows.
Hopefully, you will never need these steps. This sections covers how to recover from a complete system failure. Typically, you would boot from floppy with your Windows emergency Startup boot diskette, access your backup media, and restore the system files using the /rest option of BackEmUp.
Alternatively, you may choose to use another backup program for your initial backup of your operating system, and only use BackEmUp to manage select multiple generation backup of your data files. The multiple steps listed below to fully restore a completely destroyed Windows system may seem overly complex; but that is only because of the long file names of Windows vs the short file naming convention of the Windows Emergency Startup disk.
These steps listed below assume your operating system boot drive has been totally destroyed. Obviously, if it was another drive or just a subdirectory that has been destroyed, you can simply restore those files in one step as your operating system is still fully functional. Again, these steps below are shown assuming a total system failure.
Make sure you have created a bootable Windows startup disk. (Use the control panel, install/remove software, and select the tab to create the startup disk.)
Note, when booted using the Windows Startup disk, you are essentially running DOS. This limits the programs that you can run until you get the rest of your system functioning. It would be a good idea to boot from the startup disk before a disaster to insure you have access to the devices you need, and if not, create the diskettes necessary to access those devices.
Insure that you have drivers installed (or available) to access your backup media. For example, if you are backing up to an Iomega Zip drive, you may need to copy the guest software from the Iomega tools subdirectory: i.e. Copy the "guest.*", "*.sys", "*.ilm", and "*.mpd" files from the Iomega tools subdirectory on your hard disk to your disaster recovery floppy disk. This will allow you to access your Zip drive if your entire hard file has crashed and you have had to boot from floppy using your Windows 95 Emergency Startup disk.
If you are using compression or enciphering extensions, make sure those programs are also copied to the disaster recovery diskette along with the BackEmUp and BackInfo programs for DOS (as the Windows 95 emergency startup disk is not a fully functioning Windows 95 system) as well as the Windows version of the BackEmUp and BackInfo programs.
If you have the capability, you could considering creating an inactive volume on CD-R (or CD-RW) for the files needed to restore the operating system, and put all the tools on the CD. The Windows 98 Startup diskette has built in CD Rom support whereas under Windows 95, you would have to add those CD drivers manually.
You will boot from floppy using your Windows Startup Diskette.
Check the hard disk for errors and correct them before proceeding any further.
Depending on the type of disaster you experienced, you may have to reformat
(i.e. format c: /b ) or repartition your hard drive, or issue the
sys c: command. You should not attempt to change drive
characteristics. If you do reformat the boot partition, remember that Windows 95/98
is a DOS extension and DOS requires certain files to be in the root directory
in a specific order. The format c: /b command will reserve space
for the special files allowing BackEmUp to restore them to the
proper location. Alternatively,
the format c: /s command will put DOS files
in the correct order, but when you restore the windows components, be sure to specify
the /replaceall option so these files will be corrected for a windows
boot. This can the initial problem if you find that (1), you can't boot the drive
at all, or (2) when it boots, it only starts to a DOS command line and not the
graphica Windows GUI.
Next, using the DOS version of BackEmUp ,
you would restore (using the /rest option) the boot/hidden/system files in the root of C:\
as well as the "Windows" subdirectory.
i.e. BackEmUp C:* x: /rest /replaceall
Note the "*" is important as a selection mask is needed for restoration. Naturally, if you want only a paritial restore, enter in an appropriate file mask. See the partial restore topic for more information.
The DOS version has been tweaked to only consider the appropriate files for restoring a minimally booting Windows system.
The reason behind this DOS step at this point is we want to restore the system and bootable files in the root along with the Windows system files, but we don't want to restore the rest of the system as there could be a naming problem switching from short names to long names once we get a bootable Windows system restored. And as we've had to boot using the emergency Startup Diskette, long file name support is not available. So at this point, we are just going to restore the bare minimum to get the system to boot from the hard file to a minimally operating Windows GUI.
Note: Files in the C:\ root directory and in C:\WINDOWS are never compressed to facilitate this reload step. However, you may need those decompression program files to do a full system restore after you reboot for step 4, so make sure you have them handy.
Note, at the end of this step, there should NOT be a "Start Menu" subdirectory: i.e., the "C:\WINDOWS\STARTM~1". If so, delete it and everything below it. Why is this necessary? It appears the Windows will automatically create this long name when it boots in the next step. If it does create one, it will have a short name of "STARTM~2" which will mess up the remainder of the restore operation.
By default, the tweaked DOS version of BackEmUp should not have restored it. If it does indeed exist, you can easily do this deletion by changing to "C:\WINDOWS\COMMAND" subdirectory that you have just restored and issuing "deltree C:\WINDOWS\STARTM~1"
When the system boots from the hard drive the first time, Windows will create a dummy empty one with the correct short and long names. Then when the next step runs under full windows, it will put the correct items back under the "Start Menu" subdirectory.
After rebooting the minimally restored system, windows may struggle up the first time or two, so if Windows fails to boot, try booting again. You may have to do this a couple of times as Windows struggles to come up with this minimal file set. If you have a network card, you might want to temporarily remove it. Eventually, Windows will come up to a very minimal desktop that may appear to have missing and/or wrong images for icons. This is normal as we've only partially restored the system so far.
You will need to get to a command prompt. This can be done even though the system is only partially restored. Simply select "run" from the Start Menu and browse up to "My Computer" then the "C:" drive and select "command.com" as the program to run.
Booted up from the minimally restored operating system files on the hard disk,
run the Windows version of BackEmUp to rename the files we just restored
using short names, to the full long name version of the file.
i.e.: BackEmUp C:* x: /rest /rename
to rename (for long file names).
Note the "*" is important as a selection mask is needed for restoration. Naturally, if you want only a paritial restore, enter in an appropriate file mask. See the partial restore topic for more information.
If during this process you get a message about "invalid command interpretter",
you should issue:
BackEmUp C:\command.com x: /rest /replaceall
to restore the command interpretter again and reboot.
It is normal to receive many "file not found messages" at this step as you have only restored part of the system!
Examine the system for any incorrect long name/short name aliases and correct before going any further.
Reboot Windows again to make sure it is "clean".
Finally run the Windows version one more time to restore the remainder of the system
using the long file names.
BackEmUp C:* x: /rest
This will correct the icons on your desktop as well as fill in the programs in the Start Menu.
Note the "*" is important as a selection mask is needed for restoration. Naturally, if you want only a paritial restore, enter in an appropriate file mask. See the partial restore topic for more information.
Depending on what you have done to your system, a long file name at this point might have a different short name that it originally had due to pecularities of Windows. i.e. "A Long File Name" may have originally been equivalent to "ALONGF~2" depending on many factors, but this restore step may create the equivalence as "ALONGF~1". However, the system should function correctly.
Your desktop may appear different and many of your menus may be empty during some of these intermediate steps, but don't worry as you've only restored part of your system in the early steps.
During the boot process, you may experience error messages from the system trying to find application installed drivers that you haven't restored yet. If these are preventing you from booting to a functional MS-DOS prompt from within Windows, then repeat step 3 but restore the specific files Windows is complaining about. If the errors are non fatal, ignore them for now as you attempt to restore the rest of your application files.
Your operation system and files on the boot drive should now be fully restored. Your desktop icons should appear as what you've been used to seeing before your disaster. It may require that you actually double click on the icon for it to change correctly, but that would be a one time act.
Check the system for any errors.
For this step, you don't have to do anything complex; for example, just run
BackEmUp d:\path x: /rest where d:\path is the
drive and path of the files that were backed up, and x: is the drive of the
backup media.
OS/2 is much easier to restore as the emergency startup disks already know how to handle long file names of the HPFS file system. However, here are a few minor things to consider:
The standard utility disk set does not include some utilities that could be very useful during disaster recovery.
If you are using a ZIP drive, you'll need to include those drivers during floppy boot up sequence. As OS/2 boot from floppies includes CD Rom support, a good alternative, is to create an inactive volume on a CD R/W that not only contains the backup files, but also various utilities and backup programs mentioned.
Boot from the Utility disks. As necessary: create partitions, Boot Manager, logical volumes, format the drive, etc.
Copy the files from Utility Disk 3 to a temporary directory on the hard file and change to that temporary directory. This improves performance greatly as BackEmUp may invoke these programs several times during the course of the restore.
Make sure the TMP environment variable is set to a R/W subdirectory that has reasonable free space.
Restore the files.
i.e. BackEmUp C:* x: /rest
where x: specifies the drive (and optional path) to the control file.
Note the "*" is important as a selection mask is needed for restoration. Naturally, if you want only a paritial restore, enter in an appropriate file mask. See the partial restore topic for more information.
Remove any floppies and reboot from the hard disk.
Let's say you made some change to a bunch of files. You can restore only those files, but if they don't fit one mask definition (for example: xyz*.txt) you will have to do multiple restore commands. If however they all fit a pattern, you can restore them all at once. For example, lets say you made some programming change to a bunch of cpp files but decided that was a bad choice and you want to back those changes out. Perhaps an example will help:
Assuming the following directory structure:
d:\other
d:\project
d:\project\support
You have backed up the D: drive (i.e. backemup d:\ a: )
You have many cpp files, some in project, some in support, and some in other, but the ones you are interested in are in project only.
issue backemup d:\project\*.cpp a: /rest
This is an explanation of what that does: Although you backed up the entire d:\ drive, you only want to consider the *.cpp files in the project and lower subdirectories.
Suppose you issued instead backemup d:\*.cpp a: /S1 /rest
The /S1 option is what restricts the restore operation
to only the *.cpp files in the 1st level subdirectories. You will not restore
anything to the support subdirectory however you will restore files to the
other subdirectory as well as project.
The /replaceprompt option can give you additional flexibility. Be sure to refresh your memory on all the /replace.... options to see what will serve your purpose the best.
The options may cause you to have to think about it just a little, but you can always use the /redir path option to redirect the files to another temporary location to test out what you are really asking for. If you do specify a redirection path, it must already exist. i.e you asked for /redir C:\TEST then you should create "C:\TEST" before you start to run BackEmUp . It will create all the subdirectories under C:\TEST relative to that new logical root.
An inactive volume maintains backup files that are tracked by BackEmUp but are never automatically dropped as they age. (Note, refer to the /purge option to alter this behavior.) Thus, a CD-R is a perfect media as once they are written, they are remembered forever. However, writing a CD-R often requires special software or hardware that may not be physically on the machine you wish to backup.
Inactive volumes can also be used to consolidate multiple R/W media (such as multiple ZIP or Floppy disks) onto one larger media such as a CD-R. This can be useful if the majority of your files rarely change, but changes do happen over many directories of the disk you are backing up. For example, the boot drive of an operating system. Once installed, many of the operating system and application program files will rarely, if ever, change; but there may be INI or configuration files that change frequently. Using the inactive volume concept, one can burn a CD containing the baseline of the system and remove those backup files from the normal backup media. This can also be used where you are backing up to another drive, such as a lan alias, and wish to reclaim the space used by these seldom changing files.
In cases where you have no efficient removeable R/W media, you can initially back the system up to another spot on the hardfile, burn a CD-R with those files, then move backemup.ctl file to another media such as a floppy, and continue making incremental backups to floppy disk.
The backemup.ctl file maintains a list of volumes and the amount of free space. Each volume is logically identified by the contents of the backemup.id file that exists on each volume. That identifier is the first field of the single line that exists in the backemup.id file for that volume. So it would follow that as long as we keep each logical volume in its own subdirectory on a combined larger media, we can maintain the backup management philosophy as tradition "disk labels" are not used for identification.
In the backemup.ctl file, an inactive volume will have a -1 for the amount of free space available. Thus, this volume will never be asked for during a normal backup proceedure as it has no free space. The -1 also tells BackEmUp that there will never be obsolete files on this volume. During restore, this -1 free space signals BackEmUp to ask for an alternate path to mount this media. Thus if you had backed up to floppies as A: but have created a consolidated inactive backup media on CD-R, you can tell BackEmUp to look at your CD Rom drive to read the files. Obviously, if you were backing up to floppies, you would not want to create a CD-R for each and every floppy! As the logical volume is identified by the contents of the backemup.id file, you would consolidate many floppies into individual directories on the CD-R and point to the appropriate subdirectory when asked.
If at this point you are switching from backing up to a non removeable media to a ZIP or floppy drive, copy the backemup.id file and backemup.ctl file to the new backup media. Besure to edit the "! REMOVEABLE_MEDIA" line in the control file (or remember to use the appropriate /rm- option next time).
BackSig is a simple utility to verify the integrity of a list of files. It computes various check values (signatures) that can be manually recorded and compared. For example, if you just moved the "FILES" subdirectory onto a CD-R and want to verify the contents were written successfully, one could create a list of the files you moved (for example, use the standard DIR command with the appropriate /f or /b option and redirect the output to a file) and manually verify the check values between the 2 physical media. If the computed check values are different between the two media, then the copy was NOT created successfully.
Hint: The /a:-d /o:n /s options of the standard operating system DIR command are also handy when generating file lists.
The program takes one parameter that is the name of the file containing the
list of files. For example, create a file FILELIST.TXT and in it, list
C0000001.001
C0000002.001
Run BackSum FILELIST.TXT and you could see output resembling:
Selecting files from list: filelist.txt
Files: 2 Total Bytes: 974
Begin file contents examination phase...
Check 1, 22 7432 5914
Check 2, 26 7177 2008
Check 3, 5 6203 6552
Aux key: 9Q6V XJ4
Hint: Use standard file output redirection to save BackSig output to a file, and then use any one of a number of standard file comparison programs.