Raspberry Pi driven Dropbox alternative

As mentioned earlier I bought Raspberry Pi. After some struggles with the board shipping I finally got it almost one month after my order. Components work as expected. I went for official Debian image installation.

Lately there has been quite some publicity about privacy and security on the the public services hosting data. So I decided to minimize the Dropbox usage for file sync and go for locally hosted solution. It doesn't mean I'm out of dropbox, but rather I use it only for things I need to share outside my LAN.

My synchronization requirements

• two way sync
• non-interactive sync
• sync over ssh/local dirs (as pi works with ssh as a server out of the box)
• support for star topology - PC1..n (laptops) <-> Server (Raspberry Pi)
• keep additional dependencies to be installed on Raspberry low
• enable sync within LAN only (as I have quite some restricted internet connection with respect to data transfer quotas)
• operating on demand, rather than instantly (I want to use pi for other things as well and it's performance is not too impressive)

Alternative synchronization programs

I checked briefly only following:

• rsync
  • quite popular and maintained,
  • but not recommended for 2 way sync (as suggested on: )Two way sync with rsync.
• Unison
  • quite well known,
  • but not actively developed any more, still patches seem to be applied from time to time.
• ownCloud
  • looks actively developed
  • but compared to other 2 - eats more resources (on both sides, client and server)

To me, the unison looks like a winner. So I went for it.

Unison setup from scratch on Pi

I followed these steps:

1. [on server] changed default raspberry password via:

   passwd
   

2. [on clients and the server] set up password-less authentication for ssh connection (following the howtoforge tutorial)
3. [on clients and the server] installed unison:
   • For Fedora:

     sudo yum install unison
     

   • For (X)ubuntu/Debian:

     sudo apt-get install unison
     

4. [on clients] setup unison with ssh as a synch protocol (following the howtoforge tutorial) with the custom unison preferences file (see next section for reference).

That was basically it.

Custom setup

I created new unison preferences file:

~/.unison/unison.prf 

that has following contents:

# WATCH OUT: keep in sync across clients!

# dirs
root = /home/pb/unison
root = ssh://pi@192.168.2.109//media/My Book/pi/unison

# Work silently
batch = true

auto=true
times = true

# logging
log = true
logfile = /home/pb/.unison/unison.log

# Prevent deletion of all files if all files are locally deleted
confirmbigdeletes = true

# Check fast, don’t compare bit by bit
fastcheck = true # "the default on Unix systems", see docs

# ignore stuff
ignore = Name *~   ## ignores all files with ~ (gedit backup files)

Then I went for one more step. To synchronize via command line I issue following command on my client machines:

unison -ui text unison

To make my life easier, I created in:

~/.bashrc 

alias:

alias u-u="unison -ui text unison"

So that I can simply initiate the synchronization with:

u-u

That's it for my setup. Please note the paths as well as server IP should be adapted in case you'd be using it.

More info/inspiration

For more information, that I got inspired with, see:

• unison star topology
• File Synchronization with Unison - really detailed description
• Setting Up Unison File Synchronization Between Two Servers On Debian Squeeze
• UNIX / Linux: HowTo Use unison File Synchronizer

tail: inotify resources exhausted

I've been facing issue with tail. Once executing:

tail -f <filename_to_tail>

I got error:

tail: die Inotify-Resourcen sind erschöpft
tail: Inotify kann nicht verwendet werden, es wird auf Pollen zurückgegriffen

For those of you having english locale, you'd see:

tail: inotify resources exhausted
tail: inotify cannot be used, reverting to polling

While looking around I've found following to be valuable in understanding the problem and solving it:

• Stackoverflow: Permanent fix to tail: cannot watch `log/development.log': No space left on device
• Inotify support on Linux (instantaneous reports, no I/O load)

So the error itself means that system is getting low on inotify watches, that enable programs to monitor file/dirs changes.

To see the currently set limit (including output on my machine), I executed:

cat /proc/sys/fs/inotify/max_user_watches
8192

Well, I had to decide about the limit. So I went for 1048576 (not sure why :) ). To set it I've found 2 solutions.

Temporary solution (not preserved across restarts)

As I'm used to sudo, I went for:

sudo echo 1048576 > /proc/sys/fs/inotify/max_user_watches
-bash: /proc/sys/fs/inotify/max_user_watches: Keine Berechtigung

OK, not working. Complaining about user rights on /proc/sys/fs/inotify/max_user_watches. Then based on: How do I use sudo to redirect output to a location I don't have permission to write to? I went for:

sudo sh -c 'echo 1048576 > /proc/sys/fs/inotify/max_user_watches'

which worked, as the command proofs:

cat /proc/sys/fs/inotify/max_user_watches
1048576

The problem is that the value is reset each time I rebooted.

Permanent solution (preserved across restarts)

Adding line:

fs.inotify.max_user_watches=1048576

to:

/etc/sysctl.conf

fixed the limit value permanently (even between restarts).

Deeper analysis

However if you're interested to investigate problem in deep, it would be good idea to find out who is using the resources. There seem to be an idea at unix.stackexchange: Who's consuming my inotify resources?.

for foo in /proc/*/fd/*; do readlink -f $foo; done | grep inotify | sort | uniq -c | sort -nr

However for me it didn't show any output I could analyse to dig deep.

On the other hand I convinced myself that Dropbox is guilty here. Even though I have no proof for that (just read on some of the sources where someone suspected the same service).

Setting up Vagrant on Fedora 17

Vagrant claims it can help in setup of virtual environments and make it easy to setup new guests.

However, to play around with that, I need to have Vagrant itself installed.

As I'm on Fedora 17, I went for this guide going through:

• VirtualBox installation
• Vagrant installation and sample usage

VirtualBox installation

[FAILED] Plan A (installing the RPMFusion package)

This was supposed to be be easy.

• Searching for exact package name first

sudo yum list virtualbox

• Installing afterwards

sudo yum install VirtualBox

After the 1.st expected problem (as documented by site I followed):

VBoxManage --version
WARNING: The vboxdrv kernel module is not loaded. Either there is no module
         available for the current kernel (3.9.10-100.fc17.x86_64) or it failed to
         load. Reboot the computer or load the kernel module by executing

           '/etc/sysconfig/modules/VirtualBox.modules' (as root)

         You will not be able to start VMs until this problem is fixed.
4.2.12_RPMFusionr84980

I was not able to resolve it as suggested, with (as I faced other errors):

sudo /etc/sysconfig/modules/VirtualBox.modules
[sudo] password for pb: 
ERROR: Module vboxdrv not found.
ERROR: Module vboxnetflt not found.
ERROR: Module vboxnetadp not found.

OK, well, time to look elsewhere. I found problem similar to my at virtualbox forum, with the reference, that I should go for the official rather than rpmfusion version of the VirtualBox.

As suggested, I did the removal and installation using:

sudo yum remove VirtualBox

[OK] Plan B (installing the official VirtualBox package)

For this one I've found guide here and it worked smoothly. I've decided not to copy paste information present elsewhere I'm reffering to, so feel free to follow the steps and get back here when done. Except that I didn't follow all the steps, but rather once comming to originally failed step:

$ sudo /etc/init.d/vboxdrv setup                                                                                        
[sudo] password for pb: 
Stopping VirtualBox kernel modules                         [  OK  ]
Uninstalling old VirtualBox DKMS kernel modules            [  OK  ]
Trying to register the VirtualBox kernel modules using DKMS[  OK  ]
Starting VirtualBox kernel modules                         [  OK  ]

$ VBoxManage --version
4.2.16r86992

I was done with that (didn't go for separate VirtualBox user setup).

Vagrant installation and sample usage

Let me reffer to original steps I followed, as the rest went smoothly for me.

Afterwards I could establish ssh connection to guest, simply via:

vagrant ssh

Enjoy.

JSPWiki text strikethrough howto

How can one do the strikethrough in JSPWiki text?
Imagine following text chunk you need to partially striketrough:

this text is deprecated one this one is not

Combining information from multiple sources:

• JSPWiki bug report
• JSPWiki inline CSS example
• How to do a strikethrough with CSS

You might end up with something like this:

%%(text-decoration: line-through;) this text is deprecated one%%  this one is not

and renderred it could look something like this:

this text is deprecated one this one is not

Removing all the tables in the MySQL database

Recently I came into the problem, that I needed to clean up the MySQL database contents (to get rid of all the tables). Normally I'd go for the database drop and creation, but there are situations, where it's not feasible. The common case is that the user that has granted access to particular DB only. Can't create/recreate new one.

So I ended up (heavily inspired by some comments found here) with:

mysql -u <username> --password=<password> <database_name> -BNe "show tables" | awk '{print "drop table " $1 ";"}' | mysql -u <username> --password=<password> <database_name>

If you'd like to prevent having password stored in your shell history (and you're using file: .my.cnf to handle user/pass/database), you should go for much shorter one:

mysql -BNe "show tables" | awk '{print "drop table " $1 ";"}' | mysql

The switches explanation follows:

• -B --batch - tab delimited output, one line per record / no history
• -N --skip-column-names - do not write column names in results

As one of the users noted:

There is a very good reason to use show tables and not mysqldump. If you have a large database show tables will offer a superior performance.

simplified posts template

Those of you who have been here before have probably noticed the new look of my blog.
I like it as it brings quite some simplicity, which I consider important for the writer as well as for the readers.

The reason for the change was the fact I got stuck with things I wanted show and dynamic templates on blogger just didn't behave.

Moreover posting code snipplets is quite frequent for me and doing it the way I used to (as suggested in the post: http://peter-butkovic.blogspot.com/2012/06/code-syntax-highlighting-in-blog-posts.html) was just too much complicated.

So welcome to my simplified blog design. Hope you'll enjoy it :)