Notes about Linux software and hardware

Updated: 2005-11-26
Created: 2002

GNU/Linux and distributions (040618)

There are many meta problems and problems with GNU/Linux, but one metaproblem stands out, at least to me as I volunteer a fair bit of time to support users on Freenode's #linuxhelp channel.

This problem is that there are two immensely different levels of GNU/Linux usage: what is supported by the distribution, and what is supported if the user can install something outside the distribution. There is also of course the level where one can extend what GNU/Linux supports by writing something new, like a new application or a new device driver.

What I have found is that for ordinary users it is difficult enough to make use of what distributions offer; it is very difficult for them to install anything that is not ready-made for installation.

TBC

Particularly useful FireFox/Mozilla extensions (050422)

Listed at Texturizer.net/firefox/ and at http://extensionRoom.mozDev.org/.

Available for both Mozilla and Firefox

Firefox only

Mozilla only

Packet writing (050902)

Linux, the kernel, supports (mostly) either packet writing or something similar. Packet writing is a mode of writing to writable or rewritable CDs and DVDs that is incremental, that is block by block, instead of track by track, and with a lot less space overhead than .

Packet writing therefore allows one to handle CDs and DVDs that support it in part like a regular random access disc drive.

Types of CD and DVD media

CD and DVD media fall into thre categories depending on their rewritability:

Or put another way:

Block IO and CD and DVD media

The block IO subsystem of recent kernels can use rewritable media as if they were big floppies, writing to them directly, but this is somewhat slow; it cannot write at all to writable or erasable media, which traditionally have been written using bulk image writers like cdrecord (or Burgner) for CD-R and CD-RW or dvd+rw-tools for DVD+R, DVD+RW, DVD-R and DVD-RW.

The packet writing driver pktcdvd is a wrapper around a block device, and grabs the IO requests that would have gone to the IO device and assembles them in packets that are then sent to the CD or DVD drive in packet mode.

This has two important advantages:

However, because of various mishaps, not all drives are supported by the packet writing driver. Here are lists of drives known to work with it:

Software needed for packet writing

For packet writing these different types of software are needed:

The pktcdvd driver in the kernel
This driver comes in many different versions for different kernel versions (here is the ancient original version), and some versions have to be patched to add useful functionality or correct mistakes.
Kernel versions 2.16.12 and later have a driver that does not need to be patched; for other kernel versions follow the instructions here and here. But upgrading to the latest kernel is a lot better. The pktcdvd driver can support an old or a new API, and the new API was introduced in version 0.20 and it is important to make sure that it is available. You can check the version of the driver looking at the kernel logs for the version information it prints when the driver is initialized.
The utilities to control the driver
These utilities allow wrapping the packet driver around a block device, they are the udftools package. The name comes from UDF, the filesystem used most often for packet writing, but the package includes packet writing utilities that are not UDF specific.
If your pktcdvd driver is version 0.20 or later offering the new API, the package, if version 1.0.0b3 or earlier, must be patched as described here. Recent binary packages for udftools from most distributions already have the patch.

Preparing CDs or DVDs for packet writing

Because CD and DVD media can be used in several different modes, they often need to be physically formatted for the specific use they are put to.

CD-R, DVD+R, and DVD-R media cannot be formatted, because they are write-once; the writing mode is instructed to the drive writing to them, and the packet writing driver chooses the right mode.

MO and DVD-RAM media are as a rule ready to use, having been hard-formatted during manufacturing.

CD-RW, DVD+RW, DVD-RW can be formatted in several different modes, and it is critical that they be formatted in the right mode for use in packet writing:

Wrapping the packet writing driver

In order to use the packet writing driver it must be wrapper around the ordinary CD or DVD block driver, so reads or writes to the packet driver device are reflected onto the actual CD or DVD device.

This is done using the pktsetup utility from the udftools package, in one of two ways:

Old API
Using the old API, for pktcdvd versions before 0.20, one must create (or have created by automagic systems like devfs or udev) enough devices like /dev/pktcdvd0. This can be done for example with:
for I in 0 1 2 3; do mknod /dev/pktcdvd"$I" c 97 "$I"
Once these devices are created, the wrapping can be done with
pktsetup /dev/pktcdvd0 /dev/dvdwriter
New API
This requires the most recent udftools (version 1.0.0b3 as of 050902) with the new API patch. With the new API the packet writing device gets created with whatever name one chooses as the first argument of pktsetup, under the /dev/pktcdvd/ directory, for example with:
pktsetup 0 /dev/dvdwriter
which creates /dev/pktcddvd/0 (note the slash). The /dev/pktcdvd/ directory will also contain a control special file for the use of pktsetup, which should not be used directly.
Either way the wrapping packet writing device, for example /dev/pktcdvd0 or /dev/pktcdvd/0, will then be available for reading and writing to.

Creating and using a filesystem

Once the disc has been prepared and the packet writing deviced wrapped, it is possible to start writing to and using the disc in read or write mode.

For fully rewritable devices like MO, DVD-RAM and DVD+RW (and DVD-RW once in packet writing mode) one can in theory format the disc with and use any filesystem, including the classic ext2 and ext3.

However because even fully rewritable discs of that type tend to have limited write cycles, or very slow rewrite, it is usually best to use a filesystem designed keeping this is mind, like UDF.

Packet writing hints

Error messages in the log saying the medium is read-only
For some drives and/or media, many Linux kernel releases up to and including 2.6.16 could not do writes. This seems to have been fixed in 2.6.17.7 at least. This problem is accompanied by error messages like end_request: I/O error referring to the CD/DVD writer being used. These errors have nothing to do with packet writing, but rather with the CD/DVD writing layer.
Use of idescsi
In recent releases of the Linux 2.6 kernel there is no need of using the ATAPI/SCSI wrapper, idescsi for packet writing. You can use the IDE/ATA device name directly.

Packet writing resources

Useful IRC Linux channels, especially for newbies

Generic
Subsystem specific
Distribution specific

Partitioning suggestions for “typical” PCs

First of all there is no such thing as a “typical” PC, but for the sake of argument let's imagine one with at least 40-80GB, with a CPU with a PR of 1000-3000, and used with a GUI framework like KDE or GNOME, for web surfing, program development, documentation, and other “typical” desktop tasks.

My assumption is that most of the data will be media, such as program archives, games, music files, movies, digital photographs, and that a smaller part will be documents and program, some of them with a confidential content.

Then I would suggest something like 8-12GB for /, 2-4GB for swap, 4GB for a partition where to put your own home directory, and the rest for data storage, perhaps some of its as FAT32 (for platform independent data files like MP3s, movies, pictures, ...). Note that very large FAT32 partitions are usually a bad idea.

I prefer as Linux filesystem to have ext3 for everything, it is well shaken down and performs OK. Other plausible candidates include JFS and ReiserFS, even if in general they are less commonly supported than ext3 and a bit more complicated.

For a 160GB (152Gib) disk, I would like this layout, which for the sake of example includes dual booting with MS Windows:

Primary partitions 1 and 3:
Size Mounted on Description
12GB / ext3
12GB /fs/c, C: FAT32 for MS Windows root.
Logical partitions in the extended partition:
Size Mounted on Description
4GB swap encrypted swap
4GB /fs/home encrypted ext3 (home directory)
4GB /fs/others encrypted ext3 (other stuff, like work etc.)
30GB /fs/store ext3 for Linux downloads etc.
30GB /fs/d, D: generic data, smaller files (downloads, games)
56GB /fs/e, E: generic data, larger files (MP3, movies, ...)

I normally prefer to make sure the ext3 partitions have a block size of 1024 instead of the 4096 default, and a manually specified number of blocks per inode (about one inode per 12000 bytes for the root, and for many more bytes for archival partitions).

If you are wondering why it is important to keep FAT32 partitions small, or at least under the critical power-of-2 boundaries, it's because of growing cluster sizes.

Finally, some principles I use when devising a partitioning scheme:

Lockups with Athlon/Duron and AGP video card

There is a subtle but deadly problem with AGP cards and AMD processors if 4M virtual page mappings are enabled. This results in hard lockups. This happens fairly often for example if the 3D accelerated NVidia driver for XFree86 is used, even if it is used just in 2D mode.

The solution is to disable 4MB virtual page mappings with the boot parameter mem=nopentium. After adding this my system is no longer locking up, which it was doing especially when paging through collections of images.

Distortion with SoundBlaster Live! cards

I have recently discovered a very disappointing problem with SB Live! cards: if the volume control on a subdevice is too high, there is significant distortion of loud acute sounds, for example trumpet bits in jazz pieces.

The distortion does not happen when the master volume is at some high level; only when a subdevice volume control is at some setting higher than approximately 2/3 of the maximum. The same sound output can be obtained by either having a higher master volume with a lower subdevice volume, or viceversa. In the first case there is no distortion, in the second there is.

This is far more apparent with the Linux drivers than with the Windows drivers, and I think there is a reason for this: the Linux drivers give access to the full range of volume settings, which is from 0 to 31. The Windows drivers do only display values in percentages from 0% to 100%; my impression is that the 100% of the Windows drivers is significantly less loud than the 31 setting under Linux. This probably means that Creative are aware of the distortion and do not let the subdevice volume to be set to a high value. However I could reproduce the distortion under Windows too, at values close to 100% of the subdevice volume slider.

It is possible to mount some USB flash media cameras as disks under Linux.

Amazingly many modern cameras with a USB connection emulate a USB mass storage (flash) reader/writer instead of using some particular upload/download protocol. This means that the appear to hosts systems as USB connected removable media disk drive, a bit like an external SCSI connected ZIP disk drive.

Under Linux the can be accessed by setting up the USB subsystem and then, after the camera is connected with its SmartMedia in, loading the usb-storage module or, with Linux 2.6.10 and later, the ub modle, which takes over flash USB storage drives from usb-storage as discussed in a thread in the linux-kernel mailing list:

Either driver module will probe the USB bus, find the camera as a disk drive, and create a new virtual SCSI adapter on which the card will appear as a virtual SCSI disk (usb-storage) or a virtual disk device (ub) under /dev/ub/.

The vital detail is that as a rule the SmartMedia will contain a VFAT filesystem either in the whole volume or in the first partition of the volume; that is, if your camera's SmartMedia appears as the SCSI virtual disk /dev/sda, the VFAT filesystem will be either /dev/sda or more commonly /dev/sda1.

You can therefore access the content of any SmartMedia inserted into the camera by mounting it with a command like mount -t vfat -o noatime /dev/sda1 /fs/usb.

Because the flash memory has a limited number of writes, and is slow at writing, it is important to use -o noatime, and ideally just mount the filesystem read-only too whenever possible, and remount it read-write only when updating it.

There is now a Flash Memory HOWTO about this.

ISP terminal servers failing to do VJ header compression.

It is apparently common for some PPP terminal servers used by many ISPs for dial-in ports to have difficulty handling extended TCP options. In particular they seem to have difficulty with timestamps, but selective acknowledge, window scaling and ECN seem also to cause trouble.

Under Linux you can check this by running pppstats -w 10 while the connection is up and you are downloading; the count under VJUNC should be zero or much smaller than the one under VJCOMP.

The solution is easy under Linux: copy 0 to the tcp_timestamps (or tcp_sack, tcpd_window_scaling) file under /proc/sys/net/ipv4/.

Don't blithely disable all of the options; selective acknowledge is fairly valuable, and probably window scaling is too, for example.

TCP explicit congestion notification
causes problems with many sites or ISPs.

ECN seems to cause trouble pretty often with CISCO routers which are very common. The solution is turn ECN off under Linux by copying 0 to the tcp_ecn file under /proc/sys/net/ipv4/.

Linux and the Epson Stylus Color 680 printer

The ESC680 is a 72/144/360/720DPI printer that uses the Epson ESC/P2 command set for printers, that has been extended several times.

The ESC680 is therefore simulating a dot matrix printer (rather than a laser printer). This also means that it can print text without any driver, and even in a range of fairly good quality native fonts.

However its normal mode of operations is as a grayscale or color raster printer, in which some kind of filter takes input in some kind of format and produces a bitmaps or pixmap with inside some ESC/P2 codes that put the printer into and out of raster mode.

Things are a bit complicated as higher resolution modes are achieved by doing multiple passes of overlapping strips of pixels, so it's not quite straight bitmap or pixmap dumping.

Things are also complicated a bit by the fact that there 144 B&W ink nozzles (pins) and 48 each for each CMY color, so grayscale and colorm printing are probably handled slightly differently.

There are some filters that produce as output ESC/P2 codes and pixmaps/bitmaps for ESC printers and in particular the 680:

GhostScript
This is a filter from a close clone of PostScript to a number of output formats, including ESC/P2.
GhostScript has a generic driver for various types of dot matrix like printers, called uniprint.
The parameters that drive this generic drivers are contained in various files, for example stc800pl.upp.
In theory all one needs to print to the ESC680 is a new uniprint file with parameters specific to the printer, or more precisely to each printing mode one wants to support, which usually is a mix of resolution, color mode, dithering algorithm.
However by far the better alternative is the Gimp Print driver, which has several modes, and is rather better documented then most alternatives.
The PIPS driver for the 680 under Linux.
This driver has been developed the Japanese labs of Epson Kowa, one of the parts of Seiko/Epson. They convert from PNG image format to the ESC P/2 codes used by the 680, with several different options.
It is easy to use it as a filter, and to arrange to have stuff that need printing converted to PNG.
For simpler printing stuffs, the printer implements the full ESC P/2 command set, which means it can function as a text printer with a set of built in fonts and functions, just as if it were a dot matrix printer.
The filter comes with partial source code: the core ESC P/2 library is supplied as object code only, but the command that wraps it is supplied as source. Which is good, because it is one of the most appallingly misdesigned un-UNIX thingies one can imagine, in several different ways. For example, it is basically an image filter like netpbm, but it comes with a GTK+ GUI interface that sets system-wide (not job-specific) printing options.

Major games for Linux i386 (051126)

There are quite a few “major” games for Linux on i386, where by “major” one means with rich 3D graphics and high production values and good game play, and here I am listing my favourites in three categories: freeware, commercial and natively ported, commercial but only partly ported, requiring data files from the MS Windows edition to operate.

Several of the ported or free or freed games are available prepackaged for major distributions as a .rpm or .deb, or even with a nice installer from these very nice people.

Freeware
GL-117
Vega Strike
Commercial, native
The whole game can be bought or downloaded for GNU/Linux, separately from the MS Windows version.
Tribes 2
Unreal Tournament 2003 and 2004
America's Army (free of charge).
Enemy Territory (free of charge).
Commercial, with ported executable
For these games you buy the MS Windows version, and the GNU/Linux port is either included or can be downloaded separately.
Doom, Doom2, Heretic, Hexen
Doom3
Unreal
Quake, Quake2, Quake3, Quake4.
Castle Wolfenstein
Neverwinter Nights
Aliens Vs. Predator
Freespace 2

Some other lists:

Other free operating systems (051023)

Linux is not the only free software operating system; but for example the various BSD variants are not GPL'ed, and this reduces their utility to the free software movement (but not to Microsoft).

However, there are several other free software OSes than Linux and the BSD variants:

HelenOS
Interesting very portable new kernel, still uses all the usual POSIX/GNU style utilities.
Agnix
VSTa
AtheOS
Cosmoe
SharpOS
Exclaim
Syllable
SkyOS
ReactOS
A reimplementation of the MS Windows 32 bit APIs.
FreeDOS
OpenBeOS
Blue Eyed OS
Thix
Zeta
Visual Operating System
GUI based minimalistic OS.

TBC

Linux kernel patches (050712)

CDFS, a filesystem for structured low level access to CD discs
CDfs is a file system for Linux systems that `exports' all tracks and boot images on a CD as normal files. These files can then be mounted (e.g. for ISO and boot images), copied, played (audio and VideoCD tracks)... The primary goal for developing this file system was to `unlock' information in old ISO images.
Driver for the ZyDAS 1201 USB WiFi 802.11b chipset
This driver is part of the kernel sources since 2.6.12, but there are patches available for older 2.6 kernels, and for an older version of the driver for 2.4 kernels.
Low latency and desktop performance patches for 2.6
This is a bundle of patches designed mainly to increase the responsiveness of the 2.6 kernels for interactive applications. One advantage of this bundle is that it comes also as a set of individual patches, for pick-and-choose.
Low latency and desktop performance patches for 2.4
This is a bundle of patches designed mainly to increase the responsiveness of the 2.4 kernels for interactive applications, but there are also several new features bundled in. One advantage of this bundle is that it comes also as a set of individual patches, for pick-and-choose.
ACPI4Linux updates
ACPI support in hw and BIOS is often buggy, and under Linux is still a bit work in progress and the latest patch may contain newer workaround that may solve weird and difficult problems.
UML
UML allows running a specially patched Linux kernel as an application under another specially patched Linux kernel, in effect allowing Linux-specific virtual machines via para virtualization. It is rather less efficient than Xen but it may be preferable in some cases. It has a bit of a reputation for fragility.

Lists of Linux kernel patches and extensions