I recently had to change my “storage policy”…
A failed drive in my RAID1 set of 2 disks, and brand new disks coming up forced me to 😉 So here I am sharing two facts that have wasted my time recently. Hopefully it would help some to avoid these hassles, or at least plan more time for it.
First : I followed the NetBSD guide to setup my RAID back when I was on NetBSD3 probably. Things where a bit different from now that I have a 5 fully upgraded version, but anyway it still works. Problems is to be that I have the right same geometry for my disks in the RAID1, I did as they say on the official guide, which is copy the disklabel of disk 1 onto disk 2. and it works perfectly. Problem is : When you want to change the disk, both disklabels have same name and identifier, and if on top of that you have for some logistic reason to change the way the disks are plugged into the machine, it get much trickier to find out which drive you wanted to keep and which one you wanted to remove… So I had to “fix” my disklabels, and put different meaningful identifiers, so I could use them to properly identify which disks I wanted to pull out of my RAIDset.
Second : For my backup strategy, as my RAID set was previously “under-used”, meaning there was a lot more free space than I actually needed, I used to backup stuff inside the RAID itself. Even if it is discouraged for obvious reason, a RAID1 mirroring was enough to ensure my data, so I just wanted to be able to go dig up old configuration files and revert some setup, so just keeping the files, even in the same disk, made some sense.
Now that my disk usage is getting higher, and hard drive prices are getting lower, it s becoming worth it to get an external USB drive. I naively thought that just plugin it, and pointing my rsnapshot config to the appropriate mount point would be enough. Error.
RSnapshot uses hardlinks in order to minimize the duplicates of files into the different backups, and therefore cannot work on the preconfigured FAT partition on that drive. I expected that, so I plug that USB dirve in my ubuntu laptop, uses gparted to make a big ext2fs partition, and unplug it. From there I expected i would work. Well it didnt. As stated in the mailing list, at the time I am writing this, ext2fs has problem being mounted on the netbsd-5-0 branch, although this has been fixed in netbsd-5. I really wanted to use ext2fs for my external backup just because it would make it so easier to retrieve data on another “non-BSD” computer, I now have to upgrade to a netbsd-5 branch, rebuild everything, install the newly built OS, and reboot. Extremely easy to do since I document and script all these tasks. Just time consuming…
I have been stuck for days, when trying to update my Trac from 0.11.5 to 0.11.6 because of strange errors…
I love Trac as a free collaborative development platform, and I would definitely advise it to others, but I must admit the update has been already painful a number of times. Maybe my setup is not the simplest I could do, but anyway…
My previous Trac installation was working fine on mod_python after following the
documentation, and when I did the update, maybe because of a new version of sqlite3, maybe because of a new version of apache, or maybe because of a new version of mod_python ( more likely ) things started to go wrong…
Symptoms : blank page when trying to look at the list of the projects in my Trac environment, or when trying to view a page of any project. Seems to be that the children of my apache 2.2 webserver were dying every single time. In my /var/log/messages :
[notice] child pid 6986 exit signal Segmentation fault (11)
After a lot of rebuilding and reinstalling I managed finally to get a proper
mpinfo instead of a blank page every time I was trying to call mod_python. That was better, but still wasnt enough for trac which failed then on every page with an odd error from sqlite_backend.py :
File "usr/pkg/lib/python2.5/site-packages/trac/db/sqlite_backend.py", line 189, in get_connection
return SQLiteConnection(path, log, params)
File "usr/pkg/lib/python2.5/site-packages/trac/db/sqlite_backend.py", line 255, in __init__
SystemError: NULL result without error in PyObject_Call
After trying for hours ( days ? ) to fix that error by trying different version of everything, trying to get some error logged here and there, I finally gave up and decided, as others I believe did before me, to switch to CGI. From what I heard it seems to be a more stable way to run Trac and we do not need to bother about mod_python problems, even if it is a tiny bit slower. I just changed my apache configuration, and now everything run smoothly 🙂
My apache configuration file httpd.conf now looks like :
#For Trac CGI
ScriptAlias /trac /home/www/main/ssl/trac/trac.cgi
#to run with CGI
SetEnv TRAC_ENV_PARENT_DIR "/home/www/main/ssl/trac"
AddHandler cgi-script .cgi
#to run with mod_python
#PythonOption TracEnvParentDir /home/www/main/ssl/trac
#PythonOption TracUriRoot /trac
#SetEnv PYTHON_EGG_CACHE /home/www/main/.python-eggs
Hopefully this will avoid those with the same problem to waste too much time fighting against mod_python… These things are pretty annoying to debug.
Finally after a long struggle my development environment is up and running again 🙂
Yesterday, as I was having trouble running apache with mod_python ( lots of errors to do with pthreads ), and after a long day looking for solutions regarding mod_python without success, I decided to upgrade my old NetBSD4 to a NetBSD 5.0.1, who should have an improved thread support.
Although I was used in the past to upgrade my home server using home made scripts and docs that I keep at https://gna.org/projects/netbsd-intro , I havent done that in a long time… I updated my NetBSD source tree, and tried to build the new version while running the previous one. I failed, and after a few unsuccessful tries, I decided that doing a blank new upgrade using the installer that comes with a fresh ISO… I have been away from the mailing list probably a bit too long.
Unfortunately my server doesnt have any CD reader… so I made myself a USBdisk following the wiki. I did the upgrade on my couple of raid disks following the wiki again. But when trying to boot, it told me /boot was not present on my drive (heh ?)… It took me a while to remember that I was using FFSv2 and not v1, so I had to install the appropriate boot file : bootxx_ffsv2.
Then I rebooted, and everything was working fine. My old settings were kept, which I must admit I was a bit relieved about 🙂 My apache doesnt fails on pthreads anymore, but instead returns me an error from trac python code, which is always nicer than a plain blank page… still have to fix that…
Next step is going to completely finish the update of my system, from source and make sure all my scripts are going to work again on NetBSD 5, and that the doc is still accurate… ha lots of work, but it s always a good idea to keep track of how you install and configure your machines.
Still love NetBSD, even though I wish more people ( including myself ) had more time and resources to help them rounding the edges…
Thanks to all the people who made the release 5 possible !
I used to blog on my home server, and talk mostly about travels, posting pictures and so on… it was fun 😉
But recently I lost my passport, and lots of things went wrong, including a RAID failure on my home server that I visit once every 2 years, so I decided it wouldn’t hurt to blog somewhere else, and talk about all the geek stuff that I am used to do on my free time, such as fixing my RAID, and re-organise my backup strategy for example ;). Hopefully it will help others with similar troubles…
So here you go, I recently found that I can use a combination of cpio and ssh, to export my rsnapshot backup directory ( with hardlinks ) from a BSD machine to a Linux one !
On my NetBSD 4.0 box :
# find /home/.snapshots/ -depth -print | cpio -c -o -H ustar -v | ssh user@remotehostname "cat > snapshots.cpio"
And then on my Ubuntu box 09.10 :
$ mkdir snaps && cd snaps && cpio -imdv -H ustar -I ../snapshots.cpio
Worked like a charm 🙂
Happy me, my old server will be saved, and all backups restored as soon as I can hold a couple of new disks 🙂