simplify your daily ssh usage… even more…

On MeyerMillerSmith.com I recently posted this one:

Typing ssh commands can be very annoying but it doesn’t has to. There’s a quite nice way to simplify your daily admin life: shell scripts.

Okay, let’s say we have a dozen servers which we regularly connect to. Let’s assume that they have quite long names and we don’t want to type them every time. Surely, we could simply use Ctrl-i, but if the server name is not in the history any more, we will have to type again – and loose time.

The solution is quite easy. Create the following script connect-to-server.sh and save it i.e. under /usr/local/bin

ssh `basename $0`

Don’t forget to

chmod +x /usr/local/bin/connect-to-server.sh

Done? Great! Now we just need symlinks like this

ln -s /usr/local/bin/connect-to-server.sh \
this-is-the-name-of-the-server.my-companies-name.com

So if you want to connect to you server just enter the first letters of the server’s name and hit Tab to autocomplete, then Enter and you’re done.


Okay. This was quite cool… but not cool enough… I still had to manually symlink.

Now I wanted to extend the little script to add a new host:

#!/bin/bash
if [ $1 ]
then
 echo "adding new host: $1"
 if [ `id -u` == "0" ]
 then
 ln -s $0 `dirname $0`/$1
 else
 echo "need root privileges... using sudo..."
 sudo ln -s $0 `dirname $0`/$1
 fi
else
 echo "connecting to `basename $0`"
 ssh `basename $0`
fi

So what does it? If we don’t get a parameter, we assume that we’re in SSH mode and connect to the host represented through the filename. If we have a parameter, we assume that this is the new host to connect to and add a new host. As we are writing in /usr/local/bin we have to check whether we have root privileges. If not, we use sudo. That’s all.

Here you can download the script if you don’t want to copy & paste.

Posted in Linux, howto | Tagged , , , | Leave a comment

ProFTPD with mod_dnsbl as gentoo ebuild

Recently we realized that proftpd misses several modules that are available but not included in the standard distribution. Therefore, we created an ebuild. As time was short today, we only included mod_dnsbl. In the near future, we will add more modules.

The ebuild is available on our SVN repository: http://dev.informations-compagnie.de/svn/gentoo/net-ftp/proftpd/proftpd-1.3.2b-r1.ebuild. Please have a look at our repository intro page for instructions on how to use the additional overlay.

Posted in Linux, Software, Verschiedenes | Tagged , , , , , , , , , , , , , | Leave a comment

find out if your Apache is really serving the right hosts

From time to time it can happen that websites that used to be on your webservers are being moved by your clients without notifying you. Therefore, you will have to search for «zombie websites».

I have the following setup: There is an Apache webserver that has a lot of websites. Each website has at least two Virtual Hosts, a first for the main website and a second one fetching all the aliases and redirecting them to the first. Each website has its own config file.

So, if I want to know whether a website is still being served by my machine, I run the following:

for host in `grep "Server\(Alias\|Name\)" _config/apache/*
| awk '{ for(i=3 ; i<=NF ; i++)printf "%s\n", $i}'`; do echo
 -n `nslookup "$host" | grep -A1 Name | grep Address | awk
'{print $2}'`; echo " $host"; done


Okay, let’s go through it step by step:


for host in `grep “Server\(Alias\|Name\)” _config/apache/*

Here we extract all lines containing ServerAlias or ServerName in all config files.  The output will look like

_config/apache/foo.bar.conf: ServerName www.foo.bar
 _config/apache/foo.bar.conf: ServerAlias foo.bar fuh.bar www.foo.bar
_config/apache/example.com.conf: ServerName www.example.com
_config/apache/yah.conf: ServerAlias    example.com

As we can see, the results have a varying size. Therefore, we have to tell awk about it.


| awk ‘{ for(i=3 ; i<=NF ; i++)printf “%s\n”, $i}’`

Awk takes each line and starts to read it from the third column (each seperated with a space) until EOL. What we get is being printed with a CRLF so that we do not get example.com fuh.bar www.foo.bar but

example.com
fuh.bar
www.foo.bar

do echo -n

Now we print each of the results of the following command without a CRLF afterwards.


`nslookup “$host” | grep -A1 Name | grep Address | awk ‘{print $2}’`;

Here we perform a IP lookup of the current host, grep for a string called “Name” and let it print the following line as well as it contains the IP address we are looking for. As we just need the IP address, we do another grep on the result and pick the second line. But we don’t want the “Address” string here, so we awk it away. To make it a little bit easier to understand, I will quickly show you what these commands do:

nslookup example.com
Server:         213.133.100.100
Address:        213.133.100.100#53
Non-authoritative answer:
Name:   example.com
Address: 192.0.32.10
---
nslookup example.com | grep -A1 Name
Name:   example.com
Address: 192.0.32.10
---
nslookup example.com | grep -A1 Name | grep Address
Address: 192.0.32.10
---
nslookup example.com | grep -A1 Name | grep Address | awk '{print $2}'
192.0.32.10


echo ” $host”

Now we have an IP but we would like to know what the corresponding hostname is, so we quickly echo it.

Done. Afterwards we can see:

192.0.32.10 www.example.com
yetanotherhost.mil
Posted in Linux, Webserver, howto | Tagged , , , , , , , , , , , , , , , , | Leave a comment

Wolfram|Alpha is out

Wolfram|Alpha is out. There’s a new player on the search engine market which is the first one with an innovation since google. The scene is something between enthusiatic and at least curious. So are we. Right in this moment, they are launching their service. This is being filmed. Like the guys’n'gals from CERN did last year. Here is the cast:



Watch live video from wolframalpha on Justin.tv

Posted in Verschiedenes, allgemein | Tagged , | Leave a comment

SFTP only and SSH only OpenSSH system with gentoo

Although there are many ways workarounding the lack of security with ye olde FTP, there is a quite handy solution: use OpenSSH via SCP/SFTP to handle the file transfers.

There are many solutions which all try to restrict the access for some users. That’s not what we are trying to do. At the end, we will have two running openssh-instances. One for ssh and the other for sftp/scp.

With Gentoo, this is quite easy to do, but even for the other distros, this howto should be usable.

Here are the steps:

1. Get OpenSSH. (Should already been done at install time)

emerge openssh

2. Copy some files, make links

cp /etc/ssh /etc/ssh2 -R
cp /etc/conf.d/ssh /etc/conf.d/ssh2
ln -s /etc/init.d/sshd /etc/init.d/sshd2
ln -s /usr/sbin/sshd /usr/sbin/sshd2

3. SSH-Server

Now edit /etc/ssh/sshd_config and remove the line containing «internal subsystem». Now you can decide what to do: either bind the servers to different IP and the same ports or vice versa or both. :-) Anyway, the options for this are:

Port <portnumber>
ListenAddress <ip-address>

3. SFTP-Server

Now edit /etc/ssh2/sshd_config and keep the Port- & IP-Settings of the SSH-server in mind.

Subsystem       sftp    internal-sftp

# These lines must appear at the *end* of sshd_config
ChrootDirectory %h
ForceCommand internal-sftp

This will force every successful login to start the internal sftp server and chroot to its home directory.

Edit /etc/conf.d/sshd2

# /etc/conf.d/sshd: config file for /etc/init.d/sshd
# Where is your sshd_config file stored?
SSHD_CONFDIR=”/etc/ssh2″# Any random options you want to pass to sshd.
# See the sshd(8) manpage for more info.
SSHD_OPTS=”"
# Pid file to use (needs to be absolute path).
SSHD_PIDFILE=”/var/run/sshd2.pid”
# Path to the sshd binary (needs to be absolute path).
SSHD_BINARY=”/usr/sbin/sshd2″

4. Add SFTP to the runlevels

rc-update add ssh2 default

5. Check permissions

Make sure, that the path to each user’s home directory is being set 0755 for root:root. Otherwise, you won’t be able to log in. Let’s say, your home directory is /home/users/domains/e/example.com/t/testuser. Then, each of the path’ elements must be set to 0755 root:root. This leads to an inability of creating and removing files in the home-root. Create an incoming-files directory to get around of this.

6. EXTRA: DenyHosts just for SFTP

emerge denyhosts

edit /etc/denyhosts.conf and adapt the options to fit your needs. There is just one thing you must change:

BLOCK_SERVICE  = sshd2

If you choose to run denyhosts as daemon, I suggest to add t to the default runlevel as well. And – of course – start it.

rc-update add denyhosts default
/etc/init.d/denyhosts start

That’s all, folks! :-)

Posted in Linux, Software, howto | Tagged , , , , , , , , , | Leave a comment

rebuilding Cyrus indexes

You might know the situation, something happened, and afterwards you get error messages from Cyrus telling you that your databases just have crashed. Then, your Inbox is shown empty while the filesystem does show the correct files.

What happened?

Every Cyrus folder contains three files «cyrus.cache», «cyrus.header» and «cyrus.index». These are responsible for telling the mail client how many mails the specific folder contains and which flag has been set per mail. they are your per-folder-message-database. And that’s exactly the point why Cyrus is that much faster than any other MDA. If a mail client connects, the server just looks into these databases to serve the necessary information and just if you actually read the message, it is being loaded.
So if these databases are being corrupted, you won’t see anything but big emptiness although you might have several millions of messages physically stored in the folder.

Here are some example error messages…

Jan 12 15:02:32 host cyrus/imap[23319]: DBERROR: dbenv->open ‘/var/imap/db’ failed: Permission denied
Jan 12 15:02:32 host cyrus/imap[23319]: DBERROR: init() on berkeley
Jan 12 15:02:32 host cyrus/imap[23319]: DBERROR: reading /var/imap/db/skipstamp, assuming the worst: Permission denied
Jan 12 15:02:32 host cyrus/imap[23319]: executed
Jan 12 15:02:32 host cyrus/imap[23319]: IOERROR: opening /var/imap/mailboxes.db: Permission denied
Jan 12 15:02:32 host cyrus/imap[23319]: DBERROR: opening /var/imap/mailboxes.db: cyrusdb error
Jan 12 15:02:32 host cyrus/imap[23319]: Fatal error: can’t read mailboxes file
Jan 12 16:00:21 host cyrus/imaps[31010]: DBERROR db4: PANIC: fatal region error detected; run recovery
Jan 12 16:00:21 host cyrus/imaps[31010]: DBERROR: critical database situation
Jan 12 16:04:32 host cyrus/lmtpunix[31504]: DBERROR: opening /var/imap/deliver.db: Permission denied
Jan 12 16:04:32 host cyrus/lmtpunix[31504]: DBERROR: opening /var/imap/deliver.db: cyrusdb error
Jan 12 16:04:32 host cyrus/lmtpunix[31504]: FATAL: lmtpd: unable to init duplicate delivery database

Yes, this is exactly my problem! What’s next?

We created a little script that runs through every user’s mailbox and rebuilds the databases. the only problem is that after a successfull rebuild every message is being marked as unread. But that’s a rather small problem, we think.  :o)

Where can I get it from? I’m in a hurry!

We know you are. But before you download, please have a look at the readme or at least at the file in order to assure that the settings are correct. And don’t forget to backup. We do not grant for anything.

Download

Gentoo users can use our siczb portage overlay. Please have a look at this article to get to know how to access the overlay.

Posted in Linux, Mailserver, Software, howto | Tagged , , , , , , , , , , , , | Leave a comment

Request Tracker 3.8.2 on gentoo

We finally managed to release a package for the (currently) recent version of Best Practical’s Request Racker (rt). We added it to our layman / portage overlay. see the following article for information on how to add it to your portage tree.

Additional Notes:

  • There are several packages which are masked at the moment. We do not know what they do to your system.
  • The additional dev-perl/Encode is an rt dependency which probably will override some files originally owned by perl itself.
  • The USE flag “fastcgi” seems to be mandatory. In other words, rt won’t compile if you don’t set this flag.
  • As you might have realized by now, the whole thing is still under development so please be very careful with you system. We do not grant for anything.
Posted in Linux, Software, Webserver, allgemein | Tagged , , , , , | Leave a comment

our own gentoo portage overlay

Gentoo has, is and will be the distribution of our choice. As it is with every love, nothing is perfect. So is Gentoo. We missed the recent packages for Bestpractical’s Request Tracker. Therefore, we created our own Layman overlay. That’s a thing we were thinking of for a long time and now there was a good chance to play around. Okay, here’s what you should do in order to use the overlay:

  • edit your local layman overlay file (e.g. /usr/portage/local/layman/cache_65bd38402ac8431067b54904bd2ed2d1.xml)
  • add the following before the </layman> line:

<overlay
contact=”direktion@informations-compagnie.de”
name=”siczb”
src=”http://dev.informations-compagnie.de/svn/gentoo”
status=”unofficial”
type=”svn”>
<link>http://dev.informations-compagnie.de/svn/gentoo</link
<description>
Additional packages by saechsische Informations-Compagnie zu Berlin
</description>
</overlay>

  • do a simple layman -a siczb to add our repo to your portage tree

As usual, we do not grant for anything. So don’t blame us if your box suddenly starts burning.

Posted in Linux, Software | Tagged , , , | 1 Comment

tomcat on port 80

You might know the situation: You’d like to run tomcat from a privileged port, but the bloody thing just won’t start. Of course, it’s java-specific. Usually, a daemon gets its port from root. Not with Java. Your possibilities are therefore somewhat cruel. let us think about the options…

  1. run it as root. that’s ugly and not secure, but quick & dirty and just works.
  2. use jsvc. this will force you to mess up the funky startup script.
  3. write a C programm or use iptables. The C-thing doesn’t work (for me). iptables… mmmh…

terrible, didn’t I tell you? Well… there’s another way:

use the good-old xinetd. the way is described here (search for xinetd) and the essence comes as followed.

If you want to set up Tomcat to handle port 80 requests on your system, you’ll need to add a xinetd configuration file for this purpose. Assuming xinetd is installed with the usual paths, you can do this by adding a file (as user root) to the /etc/xinetd.d directory. Listing 1 gives a sample configuration file for Tomcat.

Listing 1. xinetd redirect configuration

# Redirects any requests on port 80
# to port 8080 (where Tomcat is listening)
service tomcat
{
socket_type = stream
protocol = tcp
user = root
wait = no
port = 80
redirect = localhost 8080
disable = no
}

After you’ve added the configuration file, you’ll need to restart xinetd to actually activate the redirection.

cool, eh?

Posted in Linux, Software, Webserver, howto | Tagged | 2 Comments

creating apache vhosts with style

We decided to have a reverse domain name structure that conatains (beside the htdocs) all vhost-related information like logs, stats and config.

The reverse structure is neccessary to get an quick overview not about the subdomains but the domains.

Additionally, we wanted to have a script which sets up the structure and creates the necessary files. If executed, the script fetches the domain name and if it starts with www a permanent redirect from example.com to www.example.com is being added as well. Moreover, we create a awstats config as well. In order to get this working properly, you will need a preconfigured awstats configuration (/etc/awstats.model.conf).

Okay. This would be the structure for the host www.example.com:

/WEBROOT/com.example.www:

conf
htdocs
logs
stats

/WEBROOT/com.example.www/conf:

awstats.www.example.com.conf
vhost.conf

/WEBROOT/com.example.www/htdocs:

/WEBROOT/com.example.www/logs:

access
error

/WEBROOT/com.example.www/stats:

As setting this up can be somewhat boring, we created a little script which can be downloaded here.

Posted in Webserver, howto | Tagged , , , , , | Leave a comment