I've upgraded these packages on quark due to security problems:
1. MySQL
2. ImageMagick
3. curl
4. enscript
5. awstats
6. emacs
7. Mailman
8. xemacs
9. python
10. sudo
11. perl
12. pdflib
13. tiff
14. unarj
15. zip
16. a2ps
17. xpdf
18. Postgres
19. Squirrelmail
The following packages weren't upgraded, but need to be:
1. wget, because of HTTP URL handling problems. It can't be removed, though, because MailScanner depends on it.
Perl was a PITA, per usual.
I've upgraded MySQL from 4.0.10 to 4.0.24 in response to multiple remote privilege escalations. I took a database dump beforehand, using the command:
# mysqldump --all-databases --password > /usr/backup/mysql.dump
I didn't need it, but it's not a bad idea to dump a database before an upgrade.
I installed portaudit on quark, which also involved upgrading the pkg_install port. We have a significant number of vulernabilities that need to be fixed.
The FreeBSD Security team reported today a vulernability with the procfs filesystem. This affects both 4-RELEASE and 5-RELEASE. I have successfully patched quarkprime and backup, and quark awaits a good time to reboot to get it patched.
I've installed a trial instance of Bugzilla on quark and quarkprime for general use by the department.
I'm using my chat server/client as a trial product in Bugzilla, and all appears to be working.
I've installed JDK 1.5.0 from Sun on the ACLs. JDK 1.4.2 is still installed but I changed the symlink /usr/local/jdk to point to /usr/local/jdk-1.5.0, and pointed /usr/local/bin/java{,c} to the binaries in /usr/local/jdk-1.5.0/bin. I tried upgrading quark's JDK, but the installer consistently crashed even after marking it as a Linux binary.
I solved a problem with lpd on quark. It was dying with the error message "flock: Resource temporarily unavilable" in its debug output, and sucking all available CPU time. I ended up rebuilding /var/spool/lpd, which solved the problem.
vi was core dumping occassionally with the error message "Resource unavailable". On a guess, I built and reinstalled vi from FreeBSD source. Hopefully that'll solve our problem.
Thanks to a tip from Hassan, I discovered that nodump acutually is not doing quite what we want it to do.
The basic story is that dump by default only honors nodump on incremental dumps rather than full dumps. I've added a "-h 0" flag to all the dumps in backup.pl to solve this problem.
To make this easier to spot in the future, I'm also working on a complete re-write of the backup.pl script to do proper error checking and urgent email notification of the admins. I'm trying to modularize it as much as possible to make future changes as easy as possible.
I've moved image's Ultra3 SCSI card into quark, to accomdate our plan (hope? dream?) of having our tape backup gear hook directly up to quark. All appears to be good. Kudos to Matt Hogan in helping me get quark in and out of the rack.
I've upgraded both quarkprime and quark to Sendmail 8.13 to allow us to take advantage of the greet_pause feature, and to be in line for bug fixes as necessary.
I've tested out SMTP AUTH and relaying, and all seems to be goodl.
I've installed strace on quark and quarkprime for debugging purposes.
I've changed the path to ispell in sqspell_setup.php from "ispell" to "/usr/local/bin/ispell". This fixes a problem where PHP can't find ispell.
I've upgraded rsync on quark and quarkprime in response to a buffer overflow regarding the way rsync handles paths.
Mic found a problem in the fortune database, which involved the installation of "potentially offensive" fortunes. The README file says that "potentially offensive" fortunes are not installed by default, but examination of the Makefile reveals that to be false. I uninstalled fortune, fixed the Makefile, and reinstalled. I've submitted a problem report to freebsd-bugs@freebsd.org regarding this.
I fixed a problem with the cvsweb configuration on quark. Something had replaced the scalar command_path with an array, which caused the CGI script to crash. I changed command_path back to a scalar and it works.
I "fixed" the arplookup complaints about a0 on quark by setting this sysctl variable:
net.link.ether.inet.log_arp_wrong_iface=0
I installed fortune from FreeBSD source for the content group.
I've created test/guest user accounts for Jim, newuser[0-19].
Here's the script I used to generate them:
#!/usr/bin/perl
for (my $n = 0; $n < 20; $n++) {
system("echo xxxxxx|pw adduser -n newuser$n -c 'Newuser $n' -g guest -e 01-Oct-2004 -w random -b /clients/users -m -k /etc/skel -h 0");
system("edquota -p newuser newuser$n");
system("cd /var/yp;make");
}
To add some extra flexibility to our authentication scheme, I've installed mod_auth_imap on quark.
Here's the basic .htaccess setup you need to do:
Auth_IMAP_Enabled on
Auth_IMAP_Authoritative on
Auth_IMAP_Server imap.server.example.com
Auth_IMAP_Log on
AuthName "blah blah blah"
I fixed the problem with pink XDM fonts by commenting out this line in quark's /usr/X11R6/lib/X11/xdm/Xresources file:
Chooser*Foreground: maroon
I fixed a virtual host problem on quark that manifested itself in requests to quark.cs.earlham.edu would go to the math department's (presumably outdated) website rather than to the CS home page.
I added a "-p" to the FETCH_CMD and FTP_CMD in /usr/ports/Mk/bsd.ports.mk to force passive mode FTP transfers. This fixes a number of hanging problems when fetching from certain FTP servers.
I setup quark to be a secondary NS for earlham.edu.
I added these lines to named.conf:
zone "28.159.IN-ADDR.ARPA" {
type slave;
file "159.28.zone";
allow-query { any; };
masters {
159.28.1.1;
};
};
zone "earlham.edu" {
type slave;
allow-query { any; };
file "earlham.edu.zone";
masters {
159.28.1.1;
};
};
I fixed a DNS issue related to the athena and bazaar clusters. The clusters were not able to resolve hostnames outside of the Earlham subnet due to an ACL preventing recursion. I added the two subnets (159.28.231.0/24 and 159.28.232.0/24) to the cs acl in named.conf on quark to solve the problem.
In response to this message, I poked around on the FreeBSD lists, and found this message. It seems to address the problem that we are having with Apache running out of memory.
The message on the list archive suggested only giving multiple restart signals to Apache. To address that, I took half of our restart signals out, so Apache only restarts when rotating the error logs of a domain.
A small warning about configuring RT:
The RT_Siteconfig.pm file is parsed by Apache when it starts or is given a SIGHUP, so be sure that there are no errors in this file, or Apache will fail to start. Restart Apache after making any changes to be sure, so Apache doesn't die at 4AM when syslog restarts it.
Per Jim's request, I created the mol mailing list for the Association for Mathematics of Language.
I've added a rule to allow incoming requests to port 587 on quark. This will allow clients to use the SMTP AUTH port rather than the normal SMTP port.
After testing out quarkprime with 4.10-STABLE, I compiled and installed 4.10-STABLE on quark. All seems to be going well.
I think we'll have some trouble upgrading the user-space tools, though, as some of them are tracked from ports. We can upgrade what we need on an individual basis, though.
I've installed RT, which is a request/ticket tracker. I'm thinking this might not be a bad idea for farming out tasks from the admin list, and also provides a nice web interface for users to track the status of their problems. I have it up here, but I haven't created any other users for it. It's got lots of options, and I'm just beginning to wade through all of them.
I also ran into some trouble during the install wrt the setup of the Postgres database. I ended up giving pgsql a temporary password, which expires tomorrow.
I'll also get SSL keys for rt up once I get a chance.
I've installed on quark for evaulation purposes. Let me know what you think.
I upgraded perl to 5.8.4 on quark. After some hocus-pocus with recompiling all the f***ing database and CGI modules, I finally got it working. Why can't this be automated?
I upgraded quark and quarkprime to the latest and greatest Squirrelmail (1.4.3a). All looks good, and we've now pulled ahead of ECS's Squirrelmail installation.
I fixed the problem we had with IP aliasing on quark. It turned out that I had forgotten to set the netmask on the aliases to 0xffffffff, which caused some weird routing problems.
I've fixed a security hole in the way DNS recursion is implemented in BIND9.
I added an acl rule for CS:
acl cs {
127.0.0.1;
192.168.0.0/24;
159.28.230.0/24;
159.28.135.0/24;
};
In the options section, I added this line:
allow-recursion { cs; };
I've patched quark and quarkprime against this vulnerability.
I have installed and configured MailScanner on quark. See this post for details.
I turned off the local display that somehow got activated, and enabled management of X terminals.
For the local display, I commented out this line in Xservers:
:0 local /usr/X11R6/bin/X
For the management of terminals, I commented out this line in xdm-config:
DisplayManager.requestPort: 0
I've upgraded quark to Apache 1.3.31 in response to several security vulnerabilities discovered in 1.3.29.
Due to problems in flexibility with mod_throttle, I've disabled that module and installed mod_bandwidth instead. It seems to be working much better at keeping excess bandwidth usage down.
Here's the setup I've used for ppckernel.org:
# As much bandwidth as we want on campus
BandWidth 159.28 0
# Limit everyone else to 75kB/s
BandWidth all 75000
# Guarentee 4MB/s on campus
MinBandWidth 159.28 4096000
# Guarente 8kB/s to everyone else
MinBandWidth all 8172
# We can be more generous to the files under 20kB
LargeFileLimit 20 100000
# Limit files between 200kB and 8MB to 50kB/s
LargeFileLimit 200 50000
# Limit anything over 8MB to 25kB/s
LargeFileLimit 8172 25000
To keep pine from complaining about self-assigned TLS certs, I've added this line to pine.conf:
smtp-server=smtp.cs.earlham.edu/tls/novalidate-cert
After doing some reading on HTTPS, I discovered that only one SSL cert can be handed out per IP address. This explained why our normal website was getting our webmail's SSL cert. To solve this, I created an IP alias for webmail, and redirected our DNS to that.
I added this line to rc.conf:
ifconfig_bge0_alias1="inet 159.28.230.251 netmask 255.255.255.255"
I then changed NameVirtualHost in httpd.conf to 159.28.230.3, and changed all the VirtualHost declarations from hostnames to IP addresses.
I had to restart saslauthd after portupgrade upgraded cyrus-sasl2.
I upgraded proftpd to 1.2.10rc1 in response to a security hole in CIDRACL.
I just found out that Apache can only handle one SSL cert per socket. This means that the webmail cert gets handed out, which is no better than the www being handed out (the other option). I'll have to regenerate certs, but that will cause some people to complain. Thoughts?
I had to regenerate the IMAP and POP3 certs on quark, because they were causing an error in Mozilla and Netscape in which the CSADMIN CN was being confused for a hostname. Changing it to imap.cs.earlham.edu/pop3.cs.earlham.edu fixed the problem.
I've added a News blog, and also created accounts for everyone in the content and pedagogical groups.
I've gotten quarkprime to mount /clients from quark over the gigabit interface.
It turned out to be a problem with the firewall rules. I had to add allow statements on both quark and quarkprime for the 192.168.0.0/24 subnet because the NFS transfers go over UDP, which is a stateless protocol.
At Hassan's request, I bumped up the minimum threshold score for spam from 5 to 8 in sa-mimedefang.cf.
I've fixed the configuration of HTTPS on quark. Previously, https://www.cs.earlham.edu went to the Squirrelmail page. Now it goes to the main page.
This involved adding NameVirtualHost parameters for 159.28.230.3:{80,443}, and adding VirtualHosts for {webmail,www}.cs.earlham.edu:{80,443}. New SSL certficates for webmail.cs.earlham.edu still have to be generated.
In the migration to the new quark, MIMEDefang stopped passing mail through SpamAssassin.
To solve that problem, I setup a separate milter for SpamAssassin. I installed the spamass-milter package, and added this line to quark.cs.earlham.edu.mc:
INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass-milter.sock, F=, T=C:15m;S:4m;R:4m;E:10m')
After restarting sendmail, everything was working fine.
(Check to make sure it starts on boot-up)
I've updated the DNS entries for quarkprime and newquarkprime. newquarkprime is now quarkprime, and the old quarkprime is now admin. For some reason, quark doesn't want to resolve admin.
I've rebuilt PostgreSQL on quark for Java support.
I used the --with-java compile-time option to enable it.
I have upgraded quark to Apache 1.3.29. Squirrelmail, PHP, PHP CGI, and mod_ssl all seem to be working fine.
This upgrade was done per this CVE vulernability report.
I've rebuilt PHP with CGI support on quark, using the port in /usr/ports/www/php4-cgi/. This went much more smoothly than last week's PHP upgrade.
Incidentally, the package database on quark has got some serious inconsistencies which prevented me from installing the package from ports. I ended up doing a "make install" in the work directory of php4-cgi, which probably isn't the best way to do it, but the only way I could think of to finish the install.
I've done some changes to the log rotation on quark. While Apache's access and error logs were being rotated once a day, with the previous days logs kept and the logs before that deleted, I have setup access logs to be kept for 90 days for analysis (rotated once a day), with error and SSL logs being kept for a week. I have also setup log rotation for the Postgres log, which has been getting quite big.
Here are the relevant lines in /etc/newsyslog.conf:
/var/log/httpd/httpd-access root:wheel 640 90 * @T00 Z
/var/log/httpd/httpd-error root:wheel 640 7 * @T00 Z
/var/log/httpd/ssl_request_log root:wheel 640 7 * @T00 Z
/var/log/httpd/ssl_engine_log root:wheel 640 7 * @T00 Z
/var/log/postgres.log postgres:production 640 7 * @T00 Z
I've enabled some more options (CGI scripts being the most important) for the ppckernel.org VirtualHosts. I've also enabled separate error and access logging for ppckernel.org.
Here are the relevant lines in httpd.conf:
ErrorLog /var/log/httpd/ppckernel-error
CustomLog /var/log/httpd/ppckernel-access combined
Options FollowSymLinks ExecCGI Indexes
I've installed a log analyzer for Apache. It can be accessed from here.
I have installed Analog/ReportMagic. Analog parses the logs, and ReportMagic uses that output to make pretty graphs. For more information, look in /usr/loca/www/analog.
Installed ifhpon Quark.
Printing seems to be fine and it's responsive for duplex and landscape printing!
The Word went over quota today, due to a 400MB Apache log file. I've rotated the log file, and set up a job in /etc/newsyslog.conf to rotate the file weekly.
Hassan and I added MySQL to PHP on quark today. We succeeded after a grueling three hours.
We learned these things:
1. Apache overwrites its SSL keys after each installation. Backups are wonderful.
2. IMAP support in mod_php is seriously broken for FreeBSD 4.6.2, even though the docs say nothing about it, and it works fine in FreeBSD 4.8.
3. Broken PHP support is likely to have nothing to do with SSL support, no matter how much it might appear that way.
4. Apache tests for the presence of /usr/local/sbin/suexec and will use it even if it's compiled with WITH_APACHE_SUEXEC=no.
I have enabled DNS aliases, Apache virtual hosts, and bandwidth throttling in both Apache and ProFTPd for the hosting of the Linux PPC kernel.
I have added these lines to httpd.conf:
===
ServerName quark.ppckernel.org
ServerAlias ppckernel.cs.earlham.edu
DocumentRoot /clients/users/ppckernel/www
ScriptAlias /cgi-bin/ /clients/users/ppckernel/www/cgi-bin/
ServerAdmin webmaster@ppckernel.org
ThrottlePolicy Speed 100 1s
ServerName www.ppckernel.org
ServerAlias ppckernel.org
DocumentRoot /clients/users/ppckernel/www
ScriptAlias /cgi-bin/ /clients/users/ppckernel/www/cgi-bin/
ServerAdmin webmaster@ppckernel.org
ThrottlePolicy Speed 100 1s
LoadModule throttle_module libexec/apache/mod_throttle.so
AddModule mod_throttle.c
ThrottlePolicy none
SetHandler throttle-status
Order deny,allow
Deny from all
Allow from .cs.earlham.edu
SetHandler throttle-me
Order deny,allow
Deny from all
Allow from .cs.earlham.edu
SetHandler throttle-me
===
I have added this line to cs.zone:
===
ppckernel IN CNAME cs.earlham.edu.
===
I have added these lines to proftpd.conf:
===
User anonppckernel
Group ppckernel
MaxClients 10
TransferRate RETR 10240:50000 group ppckernel
TransferRate STOR 20480:50000 group ppckernel
I've installed mod_throttle to Apache on quark to prepare for the hosting of the PPC kernel site. The throttling will have to be enabled in both Apache and ProFTPd for whatever directory the PPC kernel site will reside in.
Due to the bad hard drive on ntv, I changed the DNS entries for ntv and monitor to point to quark. I then created a VirtualHost entry in quark's httpd.conf to redirect requests for those hostnames to ~cricket/current/grapher.cgi.
XDM stopped working again, due to a change in the configuration files.
I restored off backup, and things started working again.
Due to the problems that have been occuring with the printers, I have put lpd into debugging mode, with output going to /var/log/lpd-errs.
The command I used was:
# lpd -D 1 -L /var/log/lpd-errs
I have upgraded OpenSSL on quark to 0.9.7c.
The full report of the security vulnerability is available here.
I have configured SpamAssassin and MIMEDefang to run on quark.
The first step is to ensure that Sendmail is compiled with milter support. Since quark already had milter support, I skipped that step.
Next, I put the following line in /etc/quark.cs.earlham.edu.mc:
INPUT_MAIL_FILTER(`mimedefang', `S=unix:/var/spool/MIMEDefang/mimedefang.sock, F=T, T=S:5m;R:5m')
I then configured MIMEDefang (which calls SpamAssassin) in /usr/local/etc/mimedefang.
The startup of MIMEDefang can be complex, so here's how to do it:
1. Fire up the SpamAssassin daemon with this command:
spamd -a -d -c
2. Start up MIMEDefang with this command:
/usr/local/etc/rc.d/rc.mimedefang start
Wait a few seconds for MIMEDefang to start up before proceeding.
3. Start up sendmail with these two commands:
sendmail -bd
sendmail -Ac -qp5m
If MIMEDefang needs a new configuration, you should use
/usr/local/etc/rc.d/rc.mimedefang reload
rather than restarting it, because sendmail depends on being able to bind
to a socket that MIMEDefang creates. If you do need to restart MIMEDefang,
it looks like the most reliable way of doing that is to stop MIMEDefang and
sendmail, start MIMEDefang, and then start sendmail.
I have gotten a better-looking theme working in Moveable Type. It turns out you have to edit the index.html template, rather than create your own. Credit goes to ECS for the template.
I have upgraded ProFTPd in response to an X-Force team security report.
Due to a buffer overflow bug in Sendmail, I have upgraded quark from 8.12.3 (ancient history) to 8.12.10.
I used /usr/src/contrib to do the upgrade, which involved moving the 8.12.10 source tree around until it matched the 8.12.3 source tree.
You can read the security report here.
A buffer overflow vulnerability in OpenSSH has been found. I have upgraded all of our systems to OpenSSH 3.7.1p2.
FreeBSD has some pecularities in its placement of configuration files and keys. Either make symlinks from /usr/local/etc to /usr/local/etc/ssh or /etc/ssh, or set the --sysconfdir manually during configuration.
Some problems were also experienced with the GNOME front-end to OpenSSH. Having never seen it used or discovered an actual purpose for it, I deinstalled it on the affected machines (RH7.3 and RH8).
Links to the security vulernabilities are available here.