Old HISTORY
1
2 This file contains some historical information on afio. It also
3 contains notes about changes in new versions.
4
5 ============Very old README file contents==============================
6
7 Afio is a better way of dealing with cpio-format archives. It is
8 generally faster than cpio, provides more diverse magnetic tape
9 options and deals somewhat gracefully with input data corruption.
10
11 Afio is quite portable. It is now running under 4.2BSD, System V
12 and even one or two UNIX-like operating systems. Please see the
13 beginning of afio.c for a brief description of the compile-time
14 configuration options.
15
16 [...]
17
18 Mark Brukhartz
19 Lachman Associates, Inc.
20 ..!{ihnp4, sun}!laidbak!mdb
21
22 [...]
23
24 =================Notes on changes in new versions==================
25
26
27 Dave Gymer (dpg@cs.nott.ac.uk) (up to version 2.3-dpg-3):
28
29 This is my port to Linux of afio. See afio.1 and README for a
30 description of what this does. It's based on work I did to port this
31 to MiNT by Eric Smith on the Atari ST, and consists mainly of bugfixes
32 and one important new feature; the use of st_rdev to indicate whether
33 a regular file was compressed by afio or not if it's name in the
34 archive ends in .z. I did this because I didn't want backups of files
35 which were already compressed (like .tar.Z archives and some of the
36 stuff in the Xfree386-1.1 distribution) to be uncompressed if I
37 restored my filesystem.
38
39 This also means that the -Z option is meaningful when listing the
40 contents of an archive; if given, any .z files are listed without the
41 .z if afio compressed them, or with the .z if they were like that when
42 afio found them.
43
44 I also ran the code through GNU Indent 1.4.
45
46 This is the third release of my version; this one knows about gzipped
47 files (.z), uses gzip instead of compress, and handles other files
48 which rarely compress well, like JPEGs.
49
50 You may want to undefine LONGZFILE if you still use the Minix fs; I
51 use xiafs so it doesn't matter. Even if you do use the Minix fs,
52 you'll only get bitten if you try to restore a compressed archive
53 without uncompressing files.
54
55 Andrew Stevens (as@prg.oxford.ac.uk) (version 2.3-dpg-4 onwards)
56
57 I've tidied the code a little to make the double-buffering work (and
58 prompt sensibly) and avoid leakage of file-handles in the compression
59 code.
60
61 This code really really needs splitting up into multiple source files!
62
63 I have tested backing up and restoring off floppies fairly thoroughly
64 (various options), but I have no the faintest idea how reliable any of
65 the other stuff is.
66
67 I have included my favourite backup script as an example of the usage
68 of afio. [This script is in the script1 subdirectory.]
69 This uses the directory /usr/adm/dump to record dumping
70 dates and a file /usr/adm/dump/DONTDUMP (example included) to
71 filter out non-dumpable areas.
72
73 The afio source as given will not attempt to compress files ending in
74 .Z .z .gz
75 .arc .gif .zip .zoo .lha .jpeg .jpg
76 .tpz .taz .tgz .tzg
77 See also the -E option in the manual page.
78
79
80 Koen Holtman (koen@win.tue.nl) (version 2.3.5 (for Linux) onwards)
81
82 Introduced new options -E, -G, -M, -w and -W.
83
84 On compressing files, afio now temporarily stores the result in memory
85 in stead of creating (potentially huge) /tmp files. This way, there
86 is no danger of the /tmp filesystem getting full during a backup, so
87 that a backup script calling afio can safely use /tmp. Afio won't use
88 more than 2 meg memory for temporary storage (also see the -M option).
89 If it runs out of memory, it calls gzip twice, the first time to
90 determine the length of the result, the second time to get the data
91 itself.
92
93 The -y and -Y options now are able to handle real shell patterns, and
94 generally work more sensible.
95 Verifying floppy disks now works on Linux.
96
97
98 Version 2.3.6 (for Linux):
99
100 Added -T option. The options -G and -T can be used to optimize the
101 backup speed/compression ratio if you have a slow machine or a fast
102 backup medium. Added -S option. Fixed some bugs in 2.3.5.
103
104
105 Version 2.3.6-dpg-1 (for Linux):
106
107 Bug fix in -F option by Dave Gymer (dpg@Cs.Nott.AC.UK).
108
109
110 Version 2.3.7:
111
112 Deleted shared memory code to reduce code clutter.
113
114 Removed `buggy gzip workaround' in compfile.c, this caused afio to
115 hang occasionally if the -f option was used. This means that afio will
116 only run with newer gzip versions. (I don't know exactly which version
117 is new enough, though gzip 1.2.3 and above certainly are. Just try
118 it.)
119
120 Edited -F code to fix `double prompting' bug.
121
122 Added more warnings to the manual.
123
124 Added some ifdefs and a few notes to make porting to other unix
125 versions easier.
126
127
128 Version 2.4:
129
130 Added -r option and small bugfix in tty handling, both by Anders
131 Baekgaard (ab@osiris.cpk.auc.dk).
132
133 Added -B option that prints the byte position of each file in the
134 archive in the -v output.
135
136 Made -F option do writing in O_SYNC mode under Linux. Thus afio will
137 notice some floppy disk write errors using Linux kernel 1.1.54 or
138 higher.
139
140 Updated manual page. Updated usage message.
141
142 Made version number shorter, though number of changes does not warrant
143 a new major version.
144
145
146 Version 2.4.1:
147
148 Fixed bug that sometimes caused unzipping to fail due to a seek on a
149 pipe (bug found and fixed by Keith Owens).
150
151 Deleted malloc() result check in meminit() to help porting, malloc(0)
152 returns NULL on some machines.
153
154 Fixed bug caused by incorrect assumptions about write(), triggered
155 under Linux if afio -iZ ... is stopped and restarted.
156
157 Now exits if a broken pipe occurs on the stderr of afio -i or the
158 stdout of afio -t and afio -rv. Thus afio -t can be used with head to
159 get a table of contents of the beginning of an archive.
160
161 Fixed bug in setting suid/sgid bits when restoring a file that does
162 not have user=root, group=root. Afio did not take into account that
163 the Linux chown() clears these bits in some cases.
164
165 Expanded semantics of -s, changed the behavior when afio gets an input
166 eof without there being an -s option given; the old behavior can be
167 gotten back by using -s 0. This change prevents bogus insert next
168 disk messages when the archive ends prematurely.
169
170 Made control file feature (added -D option).
171
172 Fixed bug in afio 2.4 that caused symbolic link info to be omitted
173 when creating an archive.
174
175 Fixed bugs in -r option.
176
177 Removed empty lines from logfile created with -L option, also fixed
178 bug in logfile creation.
179
180 Reorganized documentation files, updated manual page.
181
182
183 Version 2.4.2:
184
185 Added code to let subprocesses close archive file descriptor, needed
186 for some tape drivers when changing tapes.
187
188 Fixed bug in multivolume tape read code.
189
190 Added some portabilty stuff:
191 - Rewrote archive header scanf to be more portable.
192 - Changed CPPFLAGS into CFLAGS in Makefile.
193 - Changed ssize_t and size_t in writeall() to int.
194 - Added code to clear asb.sb_rdev in the file stat block if not a
195 device: some OSes put (garbage?) info in this field in stead of
196 clearing it and afio wants it cleared.
197
198 Added -a option to preserve atime on files read when doing -o or -r.
199 Using it will set the ctime for these files to the current time, so
200 this option _cannot_ be used together with most incremental backup
201 schemes, which rely on the ctime being preserved.
202
203 Fixed bug in -f option when used with -s.
204
205 Updated manual, added new warning to BUGS section.
206
207 Fixed tocentry() to print -B byte offset to stdout instead of stderr.
208
209 Added support for creation of UNIX sockets in restore.
210
211 Added single qoutes around rsh switches.
212
213 Added `-s 0' functionality to the -I command.
214
215 Added -z calculation to -I and -O commands.
216
217 Changed restore not to do utime() to restore time on symlinks: the
218 timestamp change gets done on the file linked to and that is not very
219 productive.
220
221 Took -N flag out of LDOPTS in Makefile. No idea what it should have
222 done.
223
224 Added `f to format' option in floppy handling code (patch by Ulrich
225 Lauther). Note that afio still does not have adequate multivolume
226 floppy support: use the tbackup program as a wrapper around afio for
227 that.
228
229 Changed default format command in Makefile from /bin/fdformat to
230 fdformat.
231
232 Changed nice(-40) to nice(-10) for subprocess in -f option.
233
234 Added options -U, -P, -Q to force compression, specify compression
235 program and pass options to the compression program, based on patch
236 file by karsten.ballueder@stud.uni-karlsruhe.de. Added script3/
237 directory to distribution.
238
239 Version 2.4.3:
240
241 Fixed bug in restore code which sometimes prevents an old file from
242 being overwritten with a hard link. Based on report by Nokubi
243 Takatsugu.
244
245 Version 2.4.4:
246
247 Fixeded more bugs in hard link handling, based on same report by
248 Nokubi Takatsugu. The inode field in the archive header only stores
249 the meast significant 16 bits, which is too little for todays
250 filesystems, yet the unpacking code assumes that these 16 bit fields
251 have unique values for each filesystem entry when handling hard links.
252 This can cause problems when unpacking files with hardlinks. Added
253 code to 1) make unpacking od old archives safer by adding more sanitiy
254 checks and 2) ensure that hardlink processing related inode fields in
255 newly produced archives are unique by inventing unique inode numbers
256 if necessary. Added -4 flag to write `extended ascii' format archives
257 with 4-byte inode fields, based on code by Nokubi Takatsugu. See
258 manpage for caveats about -4.
259
260 Incorporated some patches by Rob W. W. Hooft to speed up buffer memory
261 allocation for -Z option. Rob reported impressive speedup (due to
262 less mallocs all the time) for large files in some cases but I have
263 been unable to duplicate any dramatic speedup myself.
264
265 Fixed bug reported by Sebastian Wilhelmi: afio -o now correctly exits
266 with a fatal error and 1 exit status if there is a write error to the
267 backup medium. Added -J flag to revert to something like old
268 behavior: with -J, media write errors give a warning and writing goes
269 on, and afio finally exits with status 1. (Actual old behavior was
270 printing warning going on and exiting with status 0. Bad.)
271
272 Improved -Z reporting using patch by Ed Casas. Made some additional
273 improvements too.
274
275 Afio will now exit with status 1 if there any warnings were printed
276 during the operation (but some may be missed if -f is used).
277
278 Added a sleep(2) to lower chance of race condition in rsh processing
279 when reading remote archives with afio ... host:/file. Problems
280 reported/investigated by Rob Browning and Dirk Eddelbuettel. [Note
281 that, in spite of me correcting problems in it, remote file handling
282 is still not officially supported by this maintainer -- KH]
283
284 Incorporated some patches by Juergen Lock to make porting to (Free)BSD
285 systems easier. This should also solve an IRIX porting problem
286 reported by Mike McDonnell.
287
288 Incorporated patch by Timo Korvola to allow specification of an
289 alternative remote shell program (like ssh) for remote archives.
290
291 Added patch by Werner Koch to speficy user for remote file processing.
292 Full syntax now is `filename_without_:' for local files or
293 [user@]host[%rsh][=afio]:file for remote files. (A patch for the same
294 feature was sent by David Atkinson too.)
295
296 Added some DEC Alpha porting patches by George Brizicky. Incorporated
297 a Solaris porting patch by Erich Focht.
298
299 Added -H (promptscript) option using patch by Alexander Zangerl.
300
301 Fixed buggy handling of 0-length files which already have a .z
302 extension. Bug reported by Bob Mitchell.
303
304 Incorporated patch by Dave Gymer which adds -0 option like GNU cpio,
305 to allow input filenames to be terminated with a '\0' instead of a
306 '\n'. When used with find ... -print0, can be used to ensure that any
307 filename can be handled, even if it contains a newline. (A patch for
308 the same feature was sent by Rob Browning too.)
309
310 Lots of manpage updates, also updated BUGS section of manpage.
311
312
313 Version 2.4.5:
314
315 Fixed progress reporting and media change code to handle archives and
316 archive volumes above 2GB and 4GB correctly, based on patch by Mike
317 Black. Problems also reported by Maik Musall.
318
319 Added 'g' (gigabytes) as a possible size denotation in options taking
320 numbers, using patch by Mike Black.
321
322 Added @ option based on patch by Mike Black.
323
324 Removed bug in 2.4.4 which would report media change messages as
325 warnings about errors in the final -z count.
326
327 Added three extensions (.deb, .rpm, .bz2) to the list of those to be
328 excluded from compression by default. Patch by Dirk Eddelbuettel.
329
330 Fixed bug in code for restoring archived made with -4 option (oops!).
331 Bug found by Chris Thompson.
332
333 Incorporated patch by Bryan Henderson which fixes bug in floppy
334 prompting if -f option used (every prompt was issued twice).
335
336 Improved -H (promptscript) option using patch by Raphael Manfredi.
337 Promptscript now has tty on its stdin, and gets one more argument.
338 Included sample promptscript by Raphael Manfredi.
339
340 When writing archive without specifying -s or -s 0, added code to
341 switch to next volume if current one is full (if write() returns 0
342 bytes witten or ENOSPC error). Part of code based on patch by Raphael
343 Manfredi.
344
345 Some updates to documentation.
346
347
348 Version 2.4.6:
349
350 Fixed file closing bug in -r option which prevented tape switches when
351 verifying with some tapes. Bug reported by Tetsuya Makimura and also
352 by Alan J. Wylie.
353
354 Switched to -Wall in Makefile, modified code to suppress -Wall
355 warnings (at least with my gcc), based on patch by Holger Schurig.
356
357 Added some extensions to the list of those to be excluded from
358 compression by default. From patch by Holger Schurig.
359
360 In tests and source code checks, no year 2000 (Y2K) problems have been
361 found with afio. All afio versions back to at least 2.3.6 should have
362 no year 2000 problems, nor should existing archives produced with such
363 versions have year 2000 problems within. Current afio versions do
364 have the usual unix second counter underflow problem, which will
365 manifest itself in 2038. The date-printing afio options (-tv and -L)
366 print 4 digit years. The archive formats output by afio (both ASCII
367 and extended ASCII) use the usual unix seconds-since-1970 counts to
368 represent dates, these counts are encoded as octal numbers in a
369 wide-enough field.
370
371 Some non-Linux platforms, Sun in particular have a dev_t larger than
372 16 bits. This caused problems archiving devices on these platforms.
373 We now take this into account: if it is larger than a 16 bit number,
374 the dev_t value is re-encoded in the dev and ino parts of the header.
375 The makedev include is also updated to special-case for Sun. Based on
376 a bug report by by Guenter Steger.
377
378 Fixed bug in interference between -Z and -l options on writing an
379 archive. Fixed bug in -U, which would not force compression of hard
380 linked files. Documented internal afio limitations connected to -Z on
381 hard links and -U and hard links.
382
383 Added -1 option and revised its default for better backwards
384 compatibilty with existing backup scripts. Compatibility problem (tob
385 never going on to verify phase if afio exited 1) reported by Denis
386 Sbragion. (I recall that the problem was also reported by someone
387 else but can't find the report anymore.) Same problem, or maybe not,
388 also reported by Florin B Manolache.
389
390 Added -3 option for better compatibility with pgp. Based on bug
391 reports by Jens Getreu. And many thanks to Jens Getreu for testing
392 the -3 option! This fixes a security hole:
393
394 --snip--
395 [SECURITY] afio: security hole in 'afio -P pgp' encrypted archives
396
397 I. Description
398
399 Since version 2.4.2, the afio archiver has had an interface, the '-P
400 pgp' command line option, which can be used to pgp-encrypt the file
401 data witten to an afio archive. Following up on some bug reports, I
402 have recently discovered a security problem with this afio-pgp
403 interface: pgp encryption is not always applied in the right way.
404 This makes it possible to crack the encyption on the file data in an
405 'encrypted' archive produced using afio with the '-P pgp' option.
406
407 The security of files which were already encrypted _before_ being
408 written to the archive is not affected.
409
410 II. Impact
411
412 It is possible to cheaply crack the encryption of at least some of
413 the file data in the 'encrypted' archives produced using 'afio -P
414 pgp'. This includes archives produced using the pgp_write example
415 script included in the [pre-2.4.6] afio distribution.
416
417 III. Solution
418
419 _Existing archives_ [made with pre-2.4.6 afio versions] produced
420 with 'afio -P pgp' should really be treated with the same care
421 (against theft etc.) as unencrypted archives. If such existing
422 archives cannot be deleted or safely locked away, then encrypting
423 the _entire_ existing archive file with pgp will protect it.
424
425 --snip--
426
427 Updated scripts in script3/ to use new -3 option.
428
429 Updated manpage, added some afio command line examples. cdrecord
430 example contributed by Joachim Bergmeyer.
431
432 Added code to make -y -Y -w -W also work when creating archives.
433 Based on patch by Brian Powell.
434
435 Fixed bug in -a option, which would not restore the access time on
436 files stored with compression when creating an archive. Based on
437 patch by David Rourke.
438
439 Made some changes to ease portability to the IRIX native C compiler.
440 See also the PORTING file. Based on patch file by Brian Gunney.
441
442 Afio currently invokes gzip for compression in the -Z option.
443 Performance experiments by Ted Phelps showed that linking the zlib
444 compression library and invoking it internally will not lead to big
445 performance benefits. Though zlib is faster for very small files, it
446 turns out that invoking gzip is faster for larger files. Switching to
447 zlib would imply a major disturbance in the source. The small to
448 negative performance benefits expected do not seem to justify this, so
449 switching to zlib is not currently considered.
450
451 Made some changes to ease portability to HPUX with gcc. See also the
452 PORTING file. Based on patch file by Daniel Andersson.
453
454 Enhanced -s option based on patch by Sebastian Weber, it now can
455 accept a comma-separated list of sizes.
456
457 Updated manpage BUGS section to say that afio does not restore
458 owner/group information on symlinks. Restoring this info would be
459 possible using the relatively new `lchown' system call, but that call
460 is not supported on older linux systems and some other unix systems.
461 Problem reported by Marko Jahnke.
462
463 Added gnupg_* scripts to script3/ dir. Scripts by Jens Getreu.
464
465 Added script5/ dir and scripts, contributed by Jens Getreu.
466
467 Updated existing scripts part of SCRIPTS file.
468
469 Updated /script1 and script2/ to improve/remove /tmp file handling.
470 Based on patches by Marius Tomaschewski. These changes make the
471 scripts more resistant to funny security expliot business with
472 pre-existing /tmp files.
473
474 Added code to round down volume size to a multiple of the block size
475 in the -s option. Added -9 option to suppress this: pre-2.4.6
476 versions of afio defaulted to no rounding down. Rounding down is
477 needed for compatibility with some devices, notably ftape. Based on
478 bug report by L. C. Robinson.
479
480 Changed -H promptscript option to take running as an at or cron job
481 into account. If /dev/tty is not a connected device, afio will now
482 connect /dev/null to the stdin of the promptscript. Based on bug
483 report by Andreas Wehler.
484