backup.itcl, version 1.00.0010 - http://www.boygenius.co.uk/tivo/
The latest version of this readme can be found on
http://www.boygenius.co.uk/files/tivoweb-backup-readme.htm
Copyright (c) 2003 Andrew Whewell, portions copyright (c) 2002-2003 user 'angra' on http://www.tivocommunity.com/. Freely released under the GPL - see http://www.fsf.org/copyleft/gpl.txt. You're free to change it and / or redistribute it if you like, but if you do redistribute it then please include this readme, or a link to the readme, in your distribution.
This module is released entirely WITHOUT WARRANTY. This module makes direct writes to the database on your TiVo and it has every chance of permanently stopping your TiVo from working. If you choose to use it then you do so entirely at your own risk, I can take no responsibility whatsoever for any damage that it may cause.
The module goes to some lengths to allow backups to be made under one version of the TiVo software and restored to
another. It was written and tested on a UK TiVo version 2.5.5. The underlying proof-of-concept TCL script written by
angra was tested on version 3 and version 2 US TiVo's and was successfully used to migrate season passes and wishlists
from a version 3 US TiVo to a version 2 US TiVo.
Any other backups and restores between versions is, as far as I know, entirely untested. If you do run into problems
with the backups and restores then please check
this thread on the TiVo Community
forum and add a message there if it looks like no-one else has hit the same problem. If you do make a posting
please remember to attach the backup file to the message and mention the version of the software running
on the TiVo you are restoring to, and the type of TiVo itself. Please also take a copy of any error messages that appear
and attach those to the posting.
Finally please also check that TiVo Web is able to run correctly on the TiVo that you are trying to restore to - it
may not be compatible with recently released versions of TiVo software.
I'm assuming that you've installed TiVo Web, that you're happy with copying files onto your TiVo and that you know how to use TiVo Web. You will need to keep a note of (or find out) where on the TiVo you installed TiVo Web.
Download the latest version of the backup module - this can be found somewhere off here. The file itself will have the name 'tivoweb-backup-x_yy_zzzz.tar.gz, where x_yy_zzzz is a version number, such as 1_01_0001.
You need to copy the .gz file to the modules directory underneath the TiVo Web installation directory - so if you installed TiVo Web into /var/hack/tivoweb-tcl then you want to copy the .gz file to /var/hack/tivoweb-tcl/modules
Then you need to get to the shell prompt on the TiVo and cd to the directory where you copied the .gz file - in our example you would cd /var/hack/tivoweb-tcl/modules
If you already have the backup module installed and you are upgrading it then you should keep a backup of your
existing version, just in case you want to revert to it - this gobbet will rename it to backup.old:
mv -f backup.itcl backup.old
Next you need to decompress the module with gzip and expand the tarball - change the x_yy_zzzz part of the filename
below to the real version number for the file:
gzip -d tivoweb-backup-x_yy_zzzz.tar.gz
cpio -H tar -i < tivoweb-backup-x_yy_zzzz.tar
chmod 755 backup.itcl
Finally you need to get the new module loaded into TiVo Web. If you already had the backup module installed and you're just upgrading it then you can usually get away with doing a Quick Reload from the Restart menu in TiVo Web to get the new module running. However, if this is the first install of the module then you should stop TiVo Web altogether (use the Full Quit option in the Restart menu) and then start TiVo Web again. If you don't do this then you may get strange error messages about missing variables and procedures when you use the module.
Once the module is loaded you should have an entry on the main TiVo Web menu called 'Backup'. The main backup menu should show the version number of backup.itcl module.
Just go into the Backup option from the TiVo Web menu and choose the Backup menu entry. It asks you for a filename,
usually defaulting to a file called 'settings' in the backups directory under the TiVo Web directory. You can change
this to any valid filename if you like.
If you supply a filename that already exists then it will copy the existing file to a new extension (.old) before it
overwrites it.
Once the backup has been taken just copy it off the TiVo and store it for safe-keeping.
This is a three step process - load, configure and restore.
Load the file
If you haven't already done so, copy the backup file onto the TiVo somewhere.
Then go into the Backup option from the TiVo Web menu and choose the Restore menu entry. It asks you for the name
and location of the backup file, defaulting to the file 'settings' in the backups directory under the TiVo Web
directory. It checks to make sure that the file you supply exists and that it's a valid backup file, so you can't
really go far wrong here.
Before it loads the file the module takes a snapshot of the season passes and wishlists already on the machine. It
then compares these with the season passes and wishlists in the backup file, and figures out which ones you can
restore. It won't let you restore a season pass or wishlist if it's already there.
This snapshot is not updated if you change the season passes or wishlists on the TiVo while the backup file
is loaded. If you make changes to those items while the file is loaded then you should reload the file before actually
performing the restore.
Tell TiVo Web what you want restoring
Once the file has been loaded you'll get a screen that gives you a summary of the backup file and a summary of
what will be restored.
If the channel lineup has changed since you made the backup then you can map the old channels to your current channel
line-up in the 'channel' link - or you can leave them unmapped. Season passes on unmapped channels will not be restored.
If you have orphan series then you will get a warning and you can run a deep search for the true identifiers for
the series from the 'series' link on the summary page. Season passes for orphan series that are not 'deep searched'
will not be restored. See the 'Orphan Series' section at the end of this readme for more on what orphan series are.
By default all wishlists and season passes that can be restored will be restored. You can pick off the ones you
actually want bringing back by clicking on the 'wishlist' and 'season pass' links on the backup summary page and
picking off the relevant entries in there.
The restore module tries to detect the season passes and wishlists that are already on the TiVo so you don't
accidentally create duplicate entries, but it may not get it right all of the time. You should always check the list
of season passes that it says it will restore to make sure it's going to do exactly what you want.
Restore
Once you're happy with what is going to be restored just click the 'Restore' button at the bottom of the summary
page. The restore does not take very long, and it tells you what it is doing. Scroll down the page it produces and
keep scrolling until you see the 'Return to backup summary' link. At this point you should be done.
Note that if you don't click 'Restore' then NOTHING IS RESTORED. Loading a backup file doesn't do anything
except load the backup file. It's a three step process - load, configure and restore.
The backup module has to have a directory on the TiVo where it can write its files. By default this directory is
called backups under the main TiVo Web directory.
The backup module will usually create this directory for you. If you have installed TiVo Web on a read-only file
system (such as root) then the backup module won't be able to create the directory there and it will use /tmp instead.
There are two kinds of files written into the backup directory - the backups you create and a file called
temporary-series.tcl. This file is generated when a season pass is restored for a series that you no longer
have on your TiVo. The module recreates the series but the TiVo will not merge it into its indexes for several hours
(up to 36 hours) after the event. During this time the backup module would not be able to see the series if it relied
solely on the internal indexes within the TiVo, so it keeps a list of the series that it has added in this file. When
it detects that the series has finally been indexed by the TiVo it removes it from the file.
It goes without saying that you shouldn't change this file or remove it. Similarly you should not copy it from
the TiVo you are backing up to the TiVo you are restoring to. It is not a backup file, it is generated by the restore
and should only be used by the TiVo that created it. Only the backup file you create needs to be copied from TiVo to
TiVo.
You can tell whether a series has been indexed or not by going into the backup module and loading a backup file that
contains a season pass for the series. When the file has loaded click on the 'Series' link - if the status for the
series is 'Already on TiVo' then it has been indexed, but if it is 'Already on TiVo (added by restore but not yet
indexed)' then it is still waiting to be indexed.
When you wipe the program guide data and load on a fresh set, either by doing a guided setup or by clearing the
program guide data and to-do list, the TiVo goes off and does a huge indexing job that can last for several hours.
The TiVo will tell you when the first phase of this has finished, but it will carry on indexing in background for
several hours after this point. You can tell when it has finished by looking in /var/log/tvlog for text
along the lines of:
Mar 3 08:05:38 (none) DbGc[127]: Sweep done (eSucceeded) -- rescheduling marking in 61200 sec
You definitely shouldn't restore season passes to a TiVo until after the program guide data has been fully
indexed. I would recommend running the guided setup and then either (a) periodically checking for the Sweep done
message in tvlog, waiting a couple of hours after it appears (just to be safe) and then running the restore or (b)
just waiting for 36 hours after the guided setup before running the restore. It means a small amount of short-term
inconvenience because you have to schedule some manual recordings while you're waiting for the full index to finish,
but it's by far the safest method long term.
The backup module sometimes uses a function in TiVo Web to translate the identifier for the series into the
identifier for the series on your specific TiVo. Whether it uses this function or not depends on a number of different
circumstances. Unfortunately this function can occasionally go wrong and it can return an ID for a series that exists,
but has no program data attached to it. These series are called orphan series. If the season pass was restored to this
orphan series then it would appear in the season pass list but no programs would be scheduled to it, and it would show
as having no upcoming episodes.
This tends to happen for a little while when you have brand-new program guide data - i.e. after you have run a guided
setup or cleared and reloaded the program guide data. TiVo's that have been running for a while don't tend to have
this problem.
The restore module will detect orphan series and it will not restore a season pass for an orphan. If it sees an
orphan it gives you the opportunity to do a 'deep search' to get the correct information for the series. This is
accessible from the 'series' link on the backup file summary screen. The deep search link is not shown if no orphans
are present.
A deep search will examine each record in turn in the relevant part of the database until it finds the correct
information. It can potentially take up to half an hour, although in testing it generally took about ten minutes.
Once it finds the information you're safe to go ahead and do the restore.
A hibernating season pass is a pass for a series that is not currently being aired. The backup file records enough
information about these series to allow the restore module to recreate them on the new TiVo if required. It checks to
see if it has to rebuild the series and then, if it does, it recreates them. It has to produce a special file to
record the information about the series for the next few hours - see the 'Directories and Reinstated Series' section
above - but aside from that there is nothing terribly special (as far as the restore process is concerned) about
hibernating season passes, they should restore with no problem.
An early tester pointed out that I might be doing the module a disservice by saying that there is nothing
special about hibernating season passes. He was, of course, quite right; in that the ability to create a season pass
for a series that is not currently in the guide data is what sets the restore module apart from current TiVo and
TiVo Web functionality. You can already create season passes for programs that are currently airing. This module
lets you create season passes for programs that are not in the program guide, and may not be for some time.
This version of the module unfortunately uses a lot of memory, especially during a restore. If it uses up
all of the available memory then the TiVo will reboot.
Previous versions attempted to figure out how much memory was left and would stop before it was all used up.
However the code for this was experimental and found to be too unreliable, so it has been dropped.
Work is going on to try to reduce the amount of memory that the module needs.
The module has been tested and works fine with a season pass list of around 45 items. It is known to run out of
memory with a season pass list of around 160 items.
If you get a reboot during a restore then you can work around it by splitting the backup file into two (or more)
parts. Copy the original backup file into two new files. Then edit each file and trim down the lists of series,
season passes and wishlists. Do not trim down the list of channels or any other items.
You can also avoid the reboot by increasing the amount of memory available to TiVo Web. If you are comfortable
with editing Unix shell scripts then edit the tivoweb file in the tivoweb directory and change the
'export TIVOSH_POOLSIZE=2916352' line to 'export TIVOSH_POOLSIZE=3244032'.
Backup files produced with version 1.00.0005 onwards drastically reduce the amount of memory required on a restore
simply by only storing information for the channels used by the season passes. Previous versions stored information
about ALL of the channels. For people with 200+ channels and 15 season passes this was clearly a waste of time and
memory - most of the channel information was never used in the restore.