Sh*t my brain says and forgets about

Tag: server

Installing Icecast 2 on DreamHost VPS

Weather Underground Radio Streams

Weather Underground provided a free service to host all of these streams so you could listen into important weather bulletins over the Internet. Weather Underground was purchased by the Weather.com parent company The Weather Company in 2012 and then IBM purchased The Weather Company in 2016. Weather Underground moved all of their services over to Amazon Web Services and canned a few legacy products including the NOAA Weather Radio streams.

For years I’ve been streaming our local NOAA Weather Radio station, KEC60, over the Internet to Weather Underground. I was extremely disappointed when the weather radio service was cut off with no notice to the listeners or the people graciously providing the audio source stream to share. Uploading the audio stream from my home doesn’t use a ton of bandwidth but letting 50-100 people listen during peak times wouldn’t be practical. I have a web server running in DreamHost’s VPS cloud so why not use that and host my own streams?

DreamHost VPS

DreamHost VPS (virtual private server) is an inexpensive high-performance web hosting solution that lets you dial up and down the resources you need to run your sites. I moved over to their VPS service after having hosted a Mac mini in a local data center for years. I couldn’t beat the cost difference and DreamHost VPS is managed meaning they install security updates and mitigate any attacks on the server.

In the recent past DreamHost has removed one feature I specifically moved over to them for: having root access on your server instance. The rationale behind that change is their team needs the ability to maintain the server, install patches, inspect traffic, and do whatever other voodoo is necessary that falls under a “managed” server. The upside is I don’t have to worry about much with the server. The downside is I can’t install software on the server for all users using Aptitude, the Ubuntu system software package manager.

I need to install the streaming audio/video server Icecast 2 to be able to host my own weather radio stream for Milwaukee. Not being able to use Aptitude on my VPS instance makes this more difficult, but not impossible. The process isn’t straightforward and can be very daunting if you’ve never compiled software before on a Linux machine via the command line. I’ve documented my experience here in case you’re interested in doing something similar.

Installation

You don’t have root access on DreamHost VPS so you can’t install Icecast using apt-get. There are ways to download the apt-get packages and manually install but that has its own drawbacks and breakages. I opted to download the source code for Icecast, compile it, and install in the user directory of the site I’m hosting the stream on. By installing in the user directory I require no system-level permissions whatsoever to run Icecast.

The basic steps are:

  1. Download the source code for Icecast and a few related dependencies (libxslt, libogg, and libvorbis).
  2. Configure, compile, and install the three dependencies.
  3. Configure, compile and install Icecast.
  4. Set up the Icecast configuration.
  5. Launch Icecast.
  6. Configure Icecast to launch on startup.

Preconditions

I’m making the assumption you have a basic understanding of Linux command line operations, SSH, and how to use DreamHost’s control panel. You’ll need the following things completed before starting the process I’ve detailed:

  • Purchase a DreamHost VPS instance and have it running. (sometimes you need to be explicit with a tutorial 😝)
  • Create a shell user for your Icecast install. I suggest a separate account with DreamHost’s “Enhanced Security” turned on. This is just in case Icecast introduces any security vulnerabilities to your server and limits the access the process has to your system.
  • My VPS is using Ubuntu 12.04.5 LTS at the time of this post. At one time I did have root access on my machine and may have installed other dependencies that your instance may not have. Feel free to comment on this post if you need help installing those as well.
  • The tutorial starts with the assumption you’re logged into your account via SSH and sitting at the ~/ (home) folder location.
  • I use Bash for my shell. Other shells like Zsh should be fine but you’ll need to adjust where you edit your path.
  • The source file URLs may change and will probably be outdated shortly after I write this post. I suggest you visit the project home pages for the most recent versions when doing this yourself. You’ll also want to occasionally update Icecast as time passes to make sure you patch any security issues. This is the downfall of not being able to use Aptitude or a similar package manager.

Please note: DreamHost currently has “unlimited” bandwidth for VPS customers. This could change in the future and understand that streaming audio and video could consume a ton of bandwidth.

Download Sources

  1. Create a temporary folder:
    mkdir tmp
    cd tmp
  2. Download the source files for Icecast and the dependencies.
    wget http://downloads.xiph.org/releases/icecast/icecast-2.4.3.tar.gz
    wget https://git.gnome.org/browse/libxslt/snapshot/libxslt-1.1.29.zip
    wget http://downloads.xiph.org/releases/ogg/libogg-1.3.2.zip
    wget http://downloads.xiph.org/releases/vorbis/libvorbis-1.3.5.zip
  3. Unpack each of those downloads.
    tar xvfz *.tar.gz
    unzip *.zip

Compile All the Things

First create the usr (pronounced user) folder and add it to your path.

cd ~/
mkdir -p usr/bin
mkdir -p usr/var/log/icecast
echo 'PATH=$PATH:~/usr/bin' >> ~/.bash_profile
. ~/.bash_profile

The last command re-executes the login script so your path is updated right away.

In the following commands replace “/home/username” with the actual path of the folder you want to install everything into. In most cases you’ll just replace username with the username of the account you created.

cd ~/tmp/libxslt-1.1.29/
./autogen.sh --prefix=/home/username/usr
make install

cd ../libogg-1.3.2/
./configure --prefix=/home/username/usr
make install

cd ../libvorbis-1.3.5/
./configure --prefix=/home/username/usr
make install

cd ../icecast-2.4.3/
./configure --prefix=/home/username/usr
make install

The Autogen & Configure scripts will scan your system for dependencies and set up the configuration file that Make uses to do the actual compilation. Configure will bark if your system is missing anything important that you’ll need to additionally compile.

Configure Icecast

If everything went well you should have a fresh new Icecast installed as ~/usr/bin/icecast and a default configuration file at ~/usr/etc/icecast.xml.

I’m not going to go into too much depth on how to configure Icecast. There is a good amount of documentation on the Icecast website detailing how to get the service up and running.

At a minimum you want to replace all of the default passwords in the configuration file with something other than “hackme”.

Launch Icecast

Go ahead and launch Icecast.

cd ~/
icecast -c usr/etc/icecast.xml

This launches Icecast in interactive mode. If you exit your SSH session, Icecast shuts down. Verify that you can connect to Icecast by visiting:

http://yourserver.dreamhostps.com:8000

Change the hostname to match yours and the port number if you changed the default one in the configuration file. If everything went right you should see an Icecast status page. If you get any errors double check you launched the service properly and that you specified the server hostname correctly.

When you’re done, hit Control+C to kill Icecast. When you’re ready to keep Icecast running long-term, re-launch with -b at the end to background the process:

icecast -c usr/etc/icecast.xml -b

Launch on Startup

The last step to setting up Icecast on a DreamHost VPS is to configure it to launch when your server is rebooted. The easiest option is to use the Cron Jobs control panel provided by DreamHost.

  1. Open the Cron Jobs page in your DreamHost control panel.
  2. Click the Add New Cron Job button.
  3. Select the user you created and installed Icecast into.
  4. Enter a Title that is helpful like Icecast.
  5. Enter an e-mail address if you want log files sent to you. I left this blank.
  6. Enter the following in Command to Run (updating the path with your actual install path for Icecast):
    /home/username/usr/bin/icecast -c /home/username/usr/etc/icecast.xml -b

    The full paths are required so Cron knows exactly where Icecast is. The -b switch puts Icecast into the background.

  7. Make sure Use locking is turned on.
  8. Set When to run to Server Reboot.
  9. Click Add.

Questions?

Let me know if you have any questions in the comments on this post! I’ll probably do a follow-up post with the actual setup I’m using for streaming Weather Radio.

Check out the Milwaukee NOAA Weather Radio stream for KEC60 here:

http://milwaukeeweather.audio

Featured image courtesy of cogdog on Flickr.

Mac OS X Server Time Machine Volume Filling Too Fast

I’ve been noticing on my Mac OS X Mavericks 10.8 Server I have running in a data center has been filling up its Time Machine volume way too quickly.  The backups are continually huge and only about a week fits on the second hard drive inside of the Mac mini.  Every time the machine backed up it was taking up so much room that previous backups had to be deleted.

I searched everywhere and couldn’t find an issue.  I looked in console logs for Time Machine and couldn’t find anything definitive as well but it looked like it was something in /Library/Server.  I realized that it must be a file or set of files that are always being written to so they’re always backed up every hour or so.

I had tried some command-line tools like “du” but the usage was so many levels deep it was hard to pinpoint.  Finally I grew a brain and installed DaisyDisk on the server to see what I could find.  Within minutes I found the pinch point.  See all of the repeated pattern in the third layer from the center?

2014-08-21_07-53-41

Turns out the file /Library/Server/Xcode/Logs/xcs_proxy.log was huge and never rolled over by the logging utility.  I had been using Xcode on this server to run Bots but had since turned it off.  I still need to figure out WHY this file is being written to without the Bots service running.  Because the file is always being written to, the entire thing is backed up every single hour.

Solution:

sudo rm /Library/Server/Xcode/Logs/*

If I find the solution to the xcs_proxy issue, I’ll follow up.

Getting Macs to play with Ubuntu

I’m a fan of Unix operating systems in general.  That’s what got me interested in switching to Mac OS X because it’s Unix-based.  It was inevitable that I would eventually get a server-class machine again that wasn’t Mac-based.  The new Dell machine that I have running has Ubuntu 8.10 – a Debian-based machine which is something I’m new too.

I wanted to set up the Ubuntu machine to share files with the Macs on my network but not by using the crappy Samba protocol or even NFS.  I know both are troublesome and not as speedy on a Mac.  My only other choice was to get AFP working on the Ubuntu server and to my delight, packages exist for this.  Netatalk is an Appletalk daemon and Avahi is a Bonjour zeroconf equivalent.  Installing those packages and starting the services didn’t do it for me.  Leopard was having issues with the cleartext passwords being passed to AFPD so I went nowhere.

I did some digging and realized that OpenSSL isn’t GNU and therefore support for it in netatalk isn’t compiled in.  Not being super familar with how Debian packaging works, I looked for a guide to help me with recompiling and installing the updated netatalk package.

I found it and boy it’s an awesome guide.

http://www.kremalicious.com/2008/06/ubuntu-as-mac-file-server-and-time-machine-volume/

Take a look, try it out.  It worked perfectly me for on Ubuntu Intrepid Ibex (8.10).

Powered by WordPress & Theme by Anders Norén