How I Manage My Own Data Storage

All Posts, Computers, Linux No Comments »

I’ve been (seriously) playing with computers since I was about 12 years old.  So in the last 10 years or so, one of the things I’ve always tried to remain conscious of is how reliable (or unreliable) my computer is.  No one likes it when their computer breaks (not even someone who is talented at fixing them).  For me, this concern has been brought on by the stress of losing data that, in hindsight, may not have been all that important.  But, it was a data loss nonetheless and no one likes it when it happens.

So when I was in high school, my first strategy was to put my OS on a different drive than my user data.  I figured that the system drive would experience many more read/writes than the data drive, because of paging and other things.  So, it would be more prone to stress than a data drive that gets used less vigorously.  I kept this strategy for several years.  I was lucky enough to replace drives before they failed (as all drives do).

But, when I got to Case, I was exposed to a ridiculously fast fiber optic network that made it even easier to fill my hard drive with junk I probably don’t need.  I needed a storage solution that met a few objectives:

  • It had to be a relatively large amount available storage.
  • It had to provide some level of fault-tolerance
  • It had to run on Linux, since I was tired of switching between Linux and Windows Server 2003.  I chose Linux and I was sticking with it.

So I built a 1.2TB software RAID5 array. There’s a pic missing on that page, I’ll see if I can locate it or take another.  I also have some other items to add to that page, including benchmarks and other info.  The RAID array is terrific.  It’s survived one and a half drive failures (one legitimate drive failure and half a drive failure that I simulated by yanking a disc while it was on).  Rebuilding the array takes several hours, but that’s to be expected.  More on that when I update that post.

So of course, even with local data fault-tolerance, I want to have an off-site backup service.  At first I was using Dreamhost, but they tried to extort a bunch of money out of me to use my storage space for backups.  It was really shady.  But it did force me to look elsewhere, and I’m glad I did.  I wanted a cross-platform solution (given the heterogenous nature of my current computer collection).  So I selected JungleDisk, which uses Amazon’s S3 web service for data storage.  I pay a pitance (maybe a couple of dollars per month, if that) to store about 20GB of data.  Backups happen nightly (or however often you select).

I use JungleDisk to backup all of the documents I’ve written in college, all of the code I’ve written, all of my digital photos, my encrypted password database, and a dump of my MySQL databases.  Everything is encrypted before it even leaves my computer.  The client runs right on my file server and just works.  It’s an absolutely terrific product.  My only complaint (a feature request, really) is that I wish it would e-mail me a copy of its log file every time a backup occurs.  That way I know if it was successful and what was transferred.

I foresee keeping this setup for quite a while.

Updates: I added an extra drive to the RAID array.  A few quick terminal commands later and I’m up to 1.5TB.  Also, JungleDisk added a reporting option.  You can get an e-mail when a backup job completes but if you want details you have to pay for their monthly service.

How To: Install Stata 10 on Ubuntu 7.10

Computers, Economics, How-To's, Linux 2 Comments »

I recently had a need to install Stata 10 on Ubuntu 7.10.  It’s actually pretty easy.

1. Open a terminal
2. We’re going to have to do a few things as root so type ‘sudo su‘.
3. Type ‘mkdir -p /usr/local/stata10 && cd /usr/local/stata10
4.  Put the Stata 10 CDROM in your computer.  In the terminal, type ‘/media/cdrom0/install‘.
5.  Answer the appropriate questions in the installation wizard.
6.  When it’s done installing, it will ask you to run ‘./stinit‘.  Go ahead and type that in to the terminal.
7. After entering your licensing information, type this: ‘ln -s /usr/lib/libtiff.so.4 /usr/lib/libtiff.so.3
8. You can now start Stata 10 from the commandline with ‘/usr/local/stata10/stata‘ or ‘/usr/local/stata10/xstata‘ if you want the xwindow version.
9. Optionally, you can run ‘ln -s /usr/local/stata10/stata /usr/bin/stata‘ if you want to run Stata with just ‘stata‘ at the commandline (or you could also include the install directory in your path)

MIDS385: Edit your files on Magpie with Dreamweaver

All Posts, Computers, Linux, mids385 1 Comment »

OK, in my last post I showed you how to edit files on Magpie using a program called WinSCP3. However, that program just allowed you to transfer the files between your Magpie and your computer, but you had to use something else to actually edit the files (WinSCP3 does have a crude editing program built-in but it is not designed for what we want to do). What if you could edit the files and transfer them to and from Magpie all in the same program?

  1. Download and install Dreamweaver from the software center.
  2. Follow this guide: Windows or Mac.  The settings you want to use are:
    1. Server: magpie.case.edu
    2. user name: your user name
    3. password: your Magpie account password (I suggest checking the box for ’save password’).
    4. Check ‘Use SFTP’ This is important, if this isn’t checked, this won’t work.
    5. I strongly suggest checking the box that says “Automatically upload files to server on save.
  3. Now you should have a Site that is your Magpie account.

This will probably be my last post regarding MIDS 385 unless someone has a specific question about how to do something for the class.  If there is enough demand, I will continue to post tutorials when my schedule allows.

Ohio LinuxFest

All Posts, Computers, Linux 1 Comment »

Tomorrow morning, Colin and I will be leaving to meet up with several others at Ohio LinuxFest in Columbus.  We will be seeing people from Google, IBM, Novell, and a slew of other big-name vendors.

I will have to go through my small collection of nerdy t-shirts to find the most appropriate one.

I might try and write a post or two while I’m there, as well as upload some photos to the photo gallery.

MIDS385: A better way to edit files on Magpie?

All Posts, Computers, Linux, mids385 1 Comment »

As promised, here is a quick guide to editing files on Magpie in a friendly text editor like Notepad or Wordpad. I discourage you from using Microsoft Word to edit these files. Also, this tutorial is for Windows users. I don’t have a Mac so I can’t offer any instructions that I feel are correct enough to put my name on. However, this google search looks promising and I implore you to read through directions any way since modifying them for OS X should be pretty easy.

First, we’re going to use a free program called WinSCP3. It is a program that will let us connect to Magpie and transfer files to and from it.
Download it here and proceed to install it like you would any other software.

Second, follow these directions for using WinSCP3. Instead of the details for Columbia, use our server, magpie.case.edu, and your own username and password.

Third, once you connect, you can browse your files on your local computer on the left and the files on Magpie on the right. You can drag between the two windows. Copy your files over from your public_html folder to your local computer. Now go find where you put them and open them up in whatever text editor you like (Notepad, Wordpad, TextPad). Once you’re done editing, save your changed files, go back to WinSCP3 and copy the new versions back to the server.

Leave any comments below if you have issues. Don’t be afraid to look at the FAQ from WinSCP3’s website too. Also, I encourage you to subscribe to my blog (for free, obviously). Then, you can get an e-mail every time I update it. You can subscribe for the semester and then unsubscribe when the class is over; I won’t be offended at all :)

Also, please leave comments on the posts, otherwise I don’t know if anyone is using them (some of you have left comments already – Thank you!)!

MIDS385: How to get to Nikhil’s files

All Posts, Computers, Linux, mids385 3 Comments »

Update: Also check out my follow-up post, as promised.

OK, quick post for my classmates.

Open up PuTTY and go to magpie just like you would access your own account. Once you are logged in, type in “cd ../nxs77/public_html/mysql/class2/”. The ‘ls’ command will show you the files in that directory (type in “ls” and hit enter). If you want to copy them all to your public_html directory, type “cp * ~/public_html/”. They are now available in your home directory. If you want to edit them (surely you do, he did part of the assignment for you), type “nano [file you want to edit, without these brackets]“. For example, if I want to edit ‘index.html’ I would type “nano index.html” ‘nano’ is the same thing (essentially) as what Nikhil uses in class (‘pico’). So, a few quick hints: CTRL-O saves, CTRL-X exits, there isn’t really anything you can do with the mouse, so you will have to move the cursor around with your up/down keys on your keyboard.

Later, I will post a guide (or a link to a good one if I can find it) on how to edit the files on your computer in something you might be more familiar with (like Notepad).

Feel free to ask questions in the comment form below.

New iPods reengineered to block synching with Linux

All Posts, Computers, Linux No Comments »

I’m curious why Apple chose to do this. Why spend so much money engineering such a product only to deliberately cut out a portion of your potential consumers. It would be akin to Nike going through great efforts to design a new tennis shoe but then only sell it in one size. I guess I’ll keep my money, Apple.

New iPods reengineered to block synching with Linux – Boing Boing

Update: Well, that didn’t last long.  Hackers have, so far, found a way around the new restrictions.

Linux: Convert .bin/.cue to .ISO

All Posts, How-To's, Linux No Comments »

So what do you do when you have a “backup” that you have in .BIN,.CUE files and it needs to be mounted as an .ISO?

Bchunk is your friend here:

#> bchunk [source bin file] [source cue file] [target basename]

The first two directives are self-explanatory.  The third is the target base filename prepended by the location (if outside the current directory).  Here’s an example:

You have two files: cd-image.bin and cd-image.cue.  You’re in your home directory.  You want your newly created .ISO to be in /tmp as /tmp/cd-image.iso.  Here’s what you do:

#> bchunk cd-image.bin cd-image.cue /tmp/cd-image

Notice that I left off the .ISO at the end.  bchunk adds that for us.  I recommend adding the -v switch for verbosity.

How-To: Use your existing Windows Installation in Ubuntu

All Posts, Computers, How-To's, Linux 2 Comments »

While Linux is a great operation system, often times it’s not as easy to get things done as it is in Windows, especially if you’re used to how Windows works (or doesn’t). I found myself in a similar situation several days ago when I finally got around to installing Linux on my desktop. My two servers have been running Ubuntu for some time now, and thanks to the efforts of Colin and Lann, I have started to accumulate a decent working knowledge of the operating system. Still, sometimes I want to use Photoshop instead of GIMP or InDesign or some program that only runs in Windows. So, I followed these steps to making both systems live together peacefully using VMWare Server and Ubuntu Feisty Fawn:

First, there are some prerequisites for this:

  1. You should have a fairly speedy computer and plenty. Any virtualization software will tax your CPU and RAM. Personally, I recommend a dual-core processor, but that is by no means a requirement. As a bencmark, I have an Athlon X2 3800 and 2GB of RAM and I can hardly tell that Windows is being virtualized.
  2. You should install Linux on a separate hard drive. I have Linux installed on a 15GB hard drive I found last week, Windows was installed on another drive years ago. Remember, what we’re doing here is running Linux while keeping our existing windows installation. You don’t really need to have a separate hard drive, but it will make things much more responsive and prolong your hardware’s longevity. I have a third hard drive used for data storage.
  3. You should have some of your original install media with you. Sometimes during this process, some windows programs will become “unregistered.” For example, I have Adobe Creative Suite 3 and I had to re-register it (free) after this process because it relies on a hardware profile to stay authenticated. Since we’re changing to VMware’s virtualized hardware, we have to be conscious of programs that relied on the old setup.
  4. You should have a basic understanding of manipulating and navigating the Linux file system.
  5. Obviously, this tutorial comes caveat emptor. If you destroy your computer or its data by following this tutorial, all you will get from me is an unsympathetic “Yeah, you’ll have that.”

Great, now that we’ve got those out of the way, let’s start with our new install of Ubuntu:

  1. Let’s assume the following: You have Linux installed on /dev/hda1 and Windows installed on /dev/hdb1. This is IMPORTANT! You will have to substitute the actual locations of your installations for these devices.
  2. Your Windows installation is most likely on an NTFS partitioned drive. Ubuntu may automatically mount this drive, but it only supports read-only. However, there is a package that adds write support. Open up a terminal and run:
    sudo apt-get install ntfs-3g
  3. Unmount the Windows drive with:
    sudo umount /dev/hdb1
  4. Edit /etc/fstab and locate where your windows drive is being mounted. Change the mount type from ntfs to ntfs-3g. Also, change the options to defaults.
  5. Remount the Windows drive:
    sudo mount /dev/hdb1
  6. Install VMware:
    sudo apt-get install vmware-server
  7. Run vmware server:
    vmware
  8. Add a new virtual machine. Choose the Custom installation and configure everything appropriately. When it asks where to save the virtual server data, you can accept the default location. We will tell it more information on the next screen.
  9. Here’s the important part. When it asks whether or not to create a new virtual disk, the bottom option will say to use a current physical drive. We want that option. Select it and choose /dev/hdb and tell it to use the whole drive. Obviously, be sure to substitute the location of your own installation.
  10. At this point you should be able to boot up the virtual machine and see your familiar Windows installation.

Tips:

  1. You should either change the permissions of the mount point for your windows drive or run vmware as root (with gksu vmware) so that it can access that mount point. Otherwise, VMware will throw permissions errors.
  2. If your Linux or Windows installations are on SATA drives, then your devices will start with sd instead of hd. For example, I have a SATA hard drive located at /dev/sda.

You can leave questions or comments below.

Updates: New Toys, Linux, and more

All Posts, Computers, Linux No Comments »

This weekend, I switched over to running Ubuntu linux. I will be posting more tutorials on the random things I do so stay tuned.

Yesterday, I purchased a new LCD monitor. I was running with dual 15″ LCDs, but I decided it was time for an upgrade. Now I have a 19″ LG, which so far has been great. Plus it has a three year warranty! I also bought a really slick wireless keyboard/mouse combo and surprisingly I didn’t have to do anything in linux to get all the extra buttons working. I will be attempting to enable some other features and I will certainly post a tutorial here and on ubuntuforums.

Also, Colin informed be of WordPress’s ability to import OPML blog lists so I will be updating my list of interesting blogs (seen at the right of your screen) shortly.

WP Theme & Icons by N.Design Studio
Entries RSS Comments RSS Log in