Showing posts with label debian testing. Show all posts
Showing posts with label debian testing. Show all posts

13 January 2013

318. qmol 0.3.2: A molecular weight calculator for Linux

Over a year ago I complained about the lack of a decent molecular weight calculator in linux in general, and in Debian Testing in particular.  I eventually managed to hack together a molecular weight calculator in Python as part of an isotopic pattern calculator in Python.

However, interpreted languages like python tend to be a bit slower than compiled languages (generally not critical for a molecular weight calculator, but could be for an isotopic pattern calculator), and, perhaps more importantly, my scripts don't feature a GUI.

I vaguely remember trying to compile Kmol by Tomislav Gountchev over a year ago, and as far as I can recall it wasn't working out since it depended on packages (kde-3) that were too old.

But things are changing.

Thomas Mitterfellner has revived Kmol in the form of qmol. Since it's very recent (version 0.1 was created in November 2012, and we're now at version 0.3.2) it's not found in the Debian repos, and maybe won't for some time given that Debian Testing/Wheezy is frozen.

There are, however, a pre-built .deb file for debian squeeze/stable (and Suse, Ubuntu, Fedora etc.) -- so if you're on stable you do not have to compile. Instead go here: http://download.opensuse.org/repositories/home:/lineinthesand/

qmol is a fairly complete solution, and importantly is highly configurable while at the same time being straightforward to use. In particular I like the ability to define your own chemical groups AND the ability to run it from the command line. It's basically what I've been waiting for with the exception of the lack of an isotopic pattern calculator -- but that may come by version 1.0.

Also, the documentation -- or qmol handbook -- is quite extensive and is available under help.


Enough talking -- time for compiling.

sudo apt-get install bzip2 build-essential cmake libqtcore4 libqtgui4 qt4-qmake libqt4-dev
mkdir ~/tmp 
cd ~/tmp
wget http://downloads.sourceforge.net/project/qmol/qmol-0.3.2/qmol-0.3.2.tar.bz2
tar xvf qmol-0.3.2.tar.bz2
mkdir buildqmol
cd buildqmol/
cmake ../qmol-0.3.2
make
sudo make install

Don't worry if you get
-- Looking for Q_WS_X11 -- Looking for Q_WS_X11 - found -- Looking for Q_WS_WIN -- Looking for Q_WS_WIN - not found. -- Looking for Q_WS_QWS -- Looking for Q_WS_QWS - not found. -- Looking for Q_WS_MAC -- Looking for Q_WS_MAC - not found.

during the cmake stage.

Usage: Either run qmol from the command line:
qmol 'N(CH3)4'
N(CH3)4 = C4H12N: 74.146 g/mol C 64.80 H 16.31 N 18.89
The output format can be configured when qmol is in gui mode (Edit/Options).

or launch it by typing
qmol
A virgin window

Previous formulae aggregate at the bottom of the  window

It doesn't handle 0, but then neither does my calculator (yet)

It's very easy to define your own group -- but  only the first letter can be  upper case

Options menu -- you can format the command line output here

It works!
There are only two small things to watch out for: the inability to handle 0 (but you get an error message -- my calculator just give you an erroneous result which is arguably worse...) and the requirement that only the first letter in an abbreviation can be upper case (for reasons of ambiguity -- c.f. e.g. CHO vs C, H, O)

12 January 2013

317. Compiling samba 4.0.0 on Debian Testing/Wheezy

Not much to say about this one -- samba/SMB is only one among several linux solutions for sharing files, and probably the easiest one for sharing directories with windows computers. Samba has also been around for about as long as linux, so it has a long and interesting history.

New major versions are always exciting though. Information about Samba 4 can be found here: http://wiki.samba.org/index.php/Samba4

I won't cover configuration here -- partly because I don't know much about it -- so see the link above. To get started you might want to look at this post though: http://verahill.blogspot.com.au/2012/12/300-briefly-sharing-folder-using-samba.html

I also wrote this a long time ago, but haven't checked it for accuracy:
http://verahill.blogspot.com.au/2012/03/building-and-installing-samba-from.html

These instructions will build Samba 4.0.0 with LDAP and Cluster support.
Compilation:
sudo apt-get install build-essential libacl1-dev libattr1-dev libaio-dev libpam0g-dev kfreebsd-headers-9.0-2 libtalloc-dev python-talloc-dev libtevent-dev valgrind libsasl2-dev python2.7-dev libgnutls-dev xsltproc libctdb-dev libldap2-dev libcups2-dev python-ldb-dev ldb-tools libldb-dev checkinstall
mkdir ~/tmp 
cd ~/tmp
wget http://www.samba.org/samba/ftp/stable/samba-4.0.0.tar.gz
tar xvf samba-4.0.0.tar.gz
cd samba-4.0.0/
./configure --prefix=$HOME/.smb4
time make
sudo checkinstall
dpkg -i samba_4.0.0-1_amd64.deb

And you're done. 

Takes about 14 minutes to compile. When you run checkinstall and you're asked
Some of the files created by the installation are inside the home directory: /home You probably don't want them to be included in the package. Do you want me to list them? [n]: n Should I exclude them from the package? (Saying yes is a good idea) [n]: n

Answer no, you DO want to include them. You're getting this warning because we set a --prefix and the files aren't going to the / structure like normal programs.

The samba daemon, smbd, is in ~/.smb4/sbin/ and the configuration files are in /etc/samba/.


Here's the structure of ~/.smb4:
~/.smb4/
|-- bin
|   |-- cifsdd
|   |-- dbwrap_tool
|   |-- eventlogadm
|   |-- gentest
|   |-- ldbadd
|   |-- ldbdel
|   |-- ldbedit
|   |-- ldbmodify
|   |-- ldbrename
|   |-- ldbsearch
|   |-- locktest
|   |-- masktest
|   |-- ndrdump
|   |-- net
|   |-- nmblookup
|   |-- nmblookup4
|   |-- ntlm_auth
|   |-- oLschema2ldif
|   |-- pdbedit
|   |-- pidl
|   |-- profiles
|   |-- regdiff
|   |-- regpatch
|   |-- regshell
|   |-- regtree
|   |-- rpcclient
|   |-- samba-tool
|   |-- sharesec
|   |-- smbcacls
|   |-- smbclient
|   |-- smbclient4
|   |-- smbcontrol
|   |-- smbcquotas
|   |-- smbget
|   |-- smbpasswd
|   |-- smbspool
|   |-- smbstatus
|   |-- smbta-util
|   |-- smbtorture
|   |-- smbtree
|   |-- testparm
|   `-- wbinfo
|-- etc
|-- include
|   |-- charset.h
|   |-- core
|   |-- credentials.h
|   |-- dcerpc.h
|   |-- dcerpc_server.h
|   |-- dlinklist.h
|   |-- domain_credentials.h
|   |-- gen_ndr
|   |-- gensec.h
|   |-- ldap-util.h
|   |-- ldap_errors.h
|   |-- ldap_message.h
|   |-- ldap_ndr.h
|   |-- ldb_wrap.h
|   |-- libsmbclient.h
|   |-- lookup_sid.h
|   |-- machine_sid.h
|   |-- ndr
|   |-- ndr.h
|   |-- netapi.h
|   |-- param.h
|   |-- passdb.h
|   |-- policy.h
|   |-- read_smb.h
|   |-- registry.h
|   |-- roles.h
|   |-- rpc_common.h
|   |-- samba
|   |-- samba_util.h
|   |-- share.h
|   |-- smb2.h
|   |-- smb2_constants.h
|   |-- smb2_create_blob.h
|   |-- smb2_signing.h
|   |-- smb_cli.h
|   |-- smb_cliraw.h
|   |-- smb_common.h
|   |-- smb_composite.h
|   |-- smb_constants.h
|   |-- smb_ldap.h
|   |-- smb_raw.h
|   |-- smb_raw_interfaces.h
|   |-- smb_raw_signing.h
|   |-- smb_raw_trans2.h
|   |-- smb_request.h
|   |-- smb_seal.h
|   |-- smb_share_modes.h
|   |-- smb_signing.h
|   |-- smb_unix_ext.h
|   |-- smb_util.h
|   |-- smbconf.h
|   |-- smbldap.h
|   |-- tdr.h
|   |-- torture.h
|   |-- tsocket.h
|   |-- tsocket_internal.h
|   |-- util
|   |-- util_ldb.h
|   `-- wbclient.h
|-- lib
|   |-- auth
|   |-- bind9
|   |-- gensec
|   |-- idmap
|   |-- ldb
|   |-- libdcerpc-atsvc.so -> libdcerpc-atsvc.so.0.0.1
|   |-- libdcerpc-atsvc.so.0 -> libdcerpc-atsvc.so.0.0.1
|   |-- libdcerpc-atsvc.so.0.0.1
|   |-- libdcerpc-binding.so -> libdcerpc-binding.so.0.0.1
|   |-- libdcerpc-binding.so.0 -> libdcerpc-binding.so.0.0.1
|   |-- libdcerpc-binding.so.0.0.1
|   |-- libdcerpc-samr.so -> libdcerpc-samr.so.0.0.1
|   |-- libdcerpc-samr.so.0 -> libdcerpc-samr.so.0.0.1
|   |-- libdcerpc-samr.so.0.0.1
|   |-- libdcerpc-server.so -> libdcerpc-server.so.0.0.1
|   |-- libdcerpc-server.so.0 -> libdcerpc-server.so.0.0.1
|   |-- libdcerpc-server.so.0.0.1
|   |-- libdcerpc.so -> libdcerpc.so.0.0.1
|   |-- libdcerpc.so.0 -> libdcerpc.so.0.0.1
|   |-- libdcerpc.so.0.0.1
|   |-- libgensec.so -> libgensec.so.0.0.1
|   |-- libgensec.so.0 -> libgensec.so.0.0.1
|   |-- libgensec.so.0.0.1
|   |-- libndr-krb5pac.so -> libndr-krb5pac.so.0.0.1
|   |-- libndr-krb5pac.so.0 -> libndr-krb5pac.so.0.0.1
|   |-- libndr-krb5pac.so.0.0.1
|   |-- libndr-nbt.so -> libndr-nbt.so.0.0.1
|   |-- libndr-nbt.so.0 -> libndr-nbt.so.0.0.1
|   |-- libndr-nbt.so.0.0.1
|   |-- libndr-standard.so -> libndr-standard.so.0.0.1
|   |-- libndr-standard.so.0 -> libndr-standard.so.0.0.1
|   |-- libndr-standard.so.0.0.1
|   |-- libndr.so -> libndr.so.0.0.1
|   |-- libndr.so.0 -> libndr.so.0.0.1
|   |-- libndr.so.0.0.1
|   |-- libnetapi.so -> libnetapi.so.0
|   |-- libnetapi.so.0
|   |-- libnss_winbind.so -> libnss_winbind.so.2
|   |-- libnss_winbind.so.2
|   |-- libnss_wins.so -> libnss_wins.so.2
|   |-- libnss_wins.so.2
|   |-- libpdb.so -> libpdb.so.0
|   |-- libpdb.so.0
|   |-- libregistry.so -> libregistry.so.0.0.1
|   |-- libregistry.so.0 -> libregistry.so.0.0.1
|   |-- libregistry.so.0.0.1
|   |-- libsamba-credentials.so -> libsamba-credentials.so.0.0.1
|   |-- libsamba-credentials.so.0 -> libsamba-credentials.so.0.0.1
|   |-- libsamba-credentials.so.0.0.1
|   |-- libsamba-hostconfig.so -> libsamba-hostconfig.so.0.0.1
|   |-- libsamba-hostconfig.so.0 -> libsamba-hostconfig.so.0.0.1
|   |-- libsamba-hostconfig.so.0.0.1
|   |-- libsamba-policy.so -> libsamba-policy.so.0.0.1
|   |-- libsamba-policy.so.0 -> libsamba-policy.so.0.0.1
|   |-- libsamba-policy.so.0.0.1
|   |-- libsamba-util.so -> libsamba-util.so.0.0.1
|   |-- libsamba-util.so.0 -> libsamba-util.so.0.0.1
|   |-- libsamba-util.so.0.0.1
|   |-- libsamdb.so -> libsamdb.so.0.0.1
|   |-- libsamdb.so.0 -> libsamdb.so.0.0.1
|   |-- libsamdb.so.0.0.1
|   |-- libsmbclient-raw.so -> libsmbclient-raw.so.0.0.1
|   |-- libsmbclient-raw.so.0 -> libsmbclient-raw.so.0.0.1
|   |-- libsmbclient-raw.so.0.0.1
|   |-- libsmbclient.so -> libsmbclient.so.0.2.0
|   |-- libsmbclient.so.0 -> libsmbclient.so.0.2.0
|   |-- libsmbclient.so.0.2.0
|   |-- libsmbconf.so -> libsmbconf.so.0
|   |-- libsmbconf.so.0
|   |-- libsmbldap.so -> libsmbldap.so.0
|   |-- libsmbldap.so.0
|   |-- libsmbsharemodes.so -> libsmbsharemodes.so.0
|   |-- libsmbsharemodes.so.0
|   |-- libtevent-util.so -> libtevent-util.so.0.0.1
|   |-- libtevent-util.so.0 -> libtevent-util.so.0.0.1
|   |-- libtevent-util.so.0.0.1
|   |-- libtorture.so -> libtorture.so.0.0.1
|   |-- libtorture.so.0 -> libtorture.so.0.0.1
|   |-- libtorture.so.0.0.1
|   |-- libwbclient.so -> libwbclient.so.0.11
|   |-- libwbclient.so.0 -> libwbclient.so.0.11
|   |-- libwbclient.so.0.11
|   |-- mit_samba.so
|   |-- nss_info
|   |-- pkgconfig
|   |-- private
|   |-- process_model
|   |-- python2.7
|   |-- security
|   |-- service
|   |-- vfs
|   `-- winbind_krb5_locator.so
|-- private
|-- sbin
|   |-- nmbd
|   |-- samba
|   |-- samba_dnsupdate
|   |-- samba_kcc
|   |-- samba_spnupdate
|   |-- samba_upgradedns
|   |-- samba_upgradeprovision
|   |-- smbd
|   |-- swat
|   `-- winbindd
|-- share
|   |-- codepages
|   |-- man
|   |-- perl5
|   |-- setup
|   `-- swat
`-- var
    |-- cache
    |-- lib
    |-- lock
    |-- locks
    `-- run


11 January 2013

316. Briefly: Automated chroot/sandbox creation

EDIT:
There are plenty of reasons to use chroots, but security is not one of them.

For a practical how-to see e.g
http://pen-testing.sans.org/blog/2012/06/06/escaping-restricted-linux-shells

For a bit of yelling, see
http://yarchive.net/comp/linux/chroot.html

chroot will improve your security by creating an obstacle which may filter out some would-be crackers, but it will not make it secure by any standard. (in spite of what I may have written elsewhere on this blog).

Original post:
I've been using chroot to compile and test stuff so much lately that I figure it was time to automate the process.

Before creating your chroot you'll need a few packages:
sudo apt-get install debootstrap coreutils x11-xserver-utils


The scripts
makechroot.sh
mkdir $HOME/tmp/jail/$1 -p sudo debootstrap --arch amd64 testing $HOME/tmp/jail/$1 http://ftp.au.debian.org/debian/ sudo cp setupchroot.sh $HOME/tmp/jail/$1/

setupchroot.sh
rm /etc/apt/sources.list echo 'deb http://ftp.au.debian.org/debian/ wheezy main contrib non-free' >> /etc/apt/sources.list apt-get update apt-get install locales sudo vim echo 'export LC_ALL="C"'>>/etc/bash.bashrc echo 'export LANG="C"'>>/etc/bash.bashrc echo 'export DISPLAY=:0.0' >> /etc/bash.bashrc echo '127.0.0.1 beryllium >> /etc/hosts' source /etc/bash.bashrc adduser sandbox usermod -g sudo sandbox echo 'Defaults !tty_tickets' >> /etc/sudoers

launchchroot.sh
xhost + sudo mount -o bind /proc $1/proc sudo cp /etc/resolv.conf $1/etc/resolv.conf sudo chroot $HOME/tmp/jail/$1

How to use
To set up the chroot:
sh makechroot.sh mynewchroot
sudo chroot mynewchroot
root@beryllium:/# sh setupchroot.sh

To use the chroot:
sh launchchroot.sh mynewchroot

Once you're done with the chroot and logged out, do
sudo umount $HOME/tmp/jail/mynewchroot/proc

to unmount the /proc -- you can now delete, copy etc. the directory structure of you chroot.

315. Briefly: Compile firefox 18 on debian testing/wheezy

For no better reason than pure curiosity, here's how to build firefox 18. My regular build environment has so many packages installed these days that it's useless for working out dependencies for other people, so I tried it in a fresh chroot to hammer out the current dependencies beyond a doubt. In other words: it will work for you.

In retrospect I'm not sure if you really need the dev package for libgtk2.0.

sudo apt-get install bzip2 build-essential python zip unzip pkg-config libgtk2.0-dev gconf2 libdbus-1-dev libdbus-glib-1-dev yasm libasound2-dev libcurl4-gnutls-dev libxt-dev mesa-common-dev
mkdir ~/tmp/firefox -p
cd ~/tmp/firefox
wget ftp://ftp.mozilla.org/pub/firefox/releases/18.0/source/firefox-18.0.source.tar.bz2
tar xvf firefox-18.0.source.tar.bz2
mkdir build/
cd build/
../mozilla-release/./configure --prefix=$HOME/.firefox18
make
make install
echo "alias firefox18='$HOME/.firefox18/bin/firefox'" >> $HOME/.bashrc
source $HOME/.bashrc

Launch by typing
firefox18


09 January 2013

312. Tau + OpenMPI profiling on Debian Testing/Wheezy

Still searching for a way to easily look at the execution of parallel jobs I came across TAU: http://www.cs.uoregon.edu/Research/tau/home.php

You can download without registering, but please do register as the number of registered users tend to be important for funding and evaluation of software development in academia: http://www.cs.uoregon.edu/Research/tau/downloads.php

I'm not really sure about how to use PDT, and I've used Tau without it before without any problems.

The compilation order below is also important -- pdt won't build without libpdb.a which is generated by tau -- but you can't configure tau with -pdt if it doesn't exist.


Compiling
sudo mkdir /opt/tau
sudo chown $USER /opt/tau
cd /opt/tau

wget http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/software_werkzeuge_zur_unterstuetzung_von_programmierung_und_optimierung/otf/dateien/OTF-1.12.2salmon.tar.gz
tar xvf OTF-1.12.2salmon.tar.gz
cd /OTF-1.12.2salmon/
./configure --prefix=/opt/tau/OTF
make
make install
cd ../

wget http://tau.uoregon.edu/tau.tgz
tar xvf tau.tgz
cd tau-2.22-p1/
./configure -mpilib=/usr/lib/openmpi/lib -prefix=/opt/tau -openmp -TRACE -iowrapper -otf=/opt/tau/OTF -pthread
make install
cd ../

wget http://tau.uoregon.edu/pdt.tar.gz
tar xvf pdt.tar.gz
cd pdtoolkit-3.18.1/
./configure -prefix=/opt/tau/pdt
make
make install


cd ../tau-2.22-p1/
./configure -mpilib=/usr/lib/openmpi/lib -prefix=/opt/tau -openmp -TRACE -iowrapper -pthread -otf=/opt/tau/OTF -pdt=/opt/tau/pdt

make install


Testing
Time to try it out on something parallel.

First set the path

PATH=$PATH:/opt/tau/x86_64/bin

I used nwchem with this input file, co2.nw:
title "co nmr" geometry c 0 0 0 o 0 0 1.13 end basis * library "6-311+G*" end property shielding end dft direct grid fine mult 1 xc HFexch 0.05 slater 0.95 becke88 nonlocal 0.72 vwn_5 1 perdew91 0.81 end task dft property

and ran it using
mpirun -n 3 tau_exec nwchem co2.nw

which ends with
Total times cpu: 4.8s wall: 7.6s
It's obviously a bit too short, but will do for illustration purposes.

That generates a set of files, profile.*.0.0 -- one for each thread i.e. profile.1.0.0, profile.2.0.0 and profile.3.0.0 in this particular case. There are a lot of options for tracing, using hardware counters etc. -- see http://www.cs.uoregon.edu/Research/tau/docs/newguide/
pprof -s
Reading Profile files in profile.* FUNCTION SUMMARY (total): --------------------------------------------------------------------------------------- %Time Exclusive Inclusive #Call #Subrs Inclusive Name msec total msec usec/call --------------------------------------------------------------------------------------- 100.0 15,813 25,931 3 14276 8643959 .TAU application 18.8 4,870 4,870 10272 0 474 MPI_Barrier() 12.1 3,138 3,138 3 0 1046279 MPI_Init() 8.1 2,090 2,090 818 0 2556 MPI_Recv() 0.0 9 9 3 0 3173 MPI_Finalize() 0.0 3 3 24 0 128 MPI_Bcast() 0.0 2 2 6 0 463 MPI_Comm_dup() 0.0 1 1 790 0 2 MPI_Comm_size() 0.0 0.872 0.872 818 0 1 MPI_Send() 0.0 0.294 0.294 841 0 0 MPI_Comm_rank() 0.0 0.17 0.17 674 0 0 MPI_Get_count() 0.0 0.111 0.111 3 0 37 MPI_Comm_free() 0.0 0.026 0.026 3 0 9 MPI_Errhandler_set() 0.0 0.024 0.024 6 0 4 MPI_Group_rank() 0.0 0.02 0.02 6 0 3 MPI_Comm_compare() 0.0 0.015 0.015 4 0 4 MPI_Comm_group() 0.0 0.008 0.008 4 0 2 MPI_Group_size() 0.0 0.004 0.004 1 0 4 MPI_Group_translate_ranks() FUNCTION SUMMARY (mean): --------------------------------------------------------------------------------------- %Time Exclusive Inclusive #Call #Subrs Inclusive Name msec total msec usec/call --------------------------------------------------------------------------------------- 100.0 5,271 8,643 1 4758.67 8643959 .TAU application 18.8 1,623 1,623 3424 0 474 MPI_Barrier() 12.1 1,046 1,046 1 0 1046279 MPI_Init() 8.1 696 696 272.667 0 2556 MPI_Recv() 0.0 3 3 1 0 3173 MPI_Finalize() 0.0 1 1 8 0 128 MPI_Bcast() 0.0 0.926 0.926 2 0 463 MPI_Comm_dup() 0.0 0.436 0.436 263.333 0 2 MPI_Comm_size() 0.0 0.291 0.291 272.667 0 1 MPI_Send() 0.0 0.098 0.098 280.333 0 0 MPI_Comm_rank() 0.0 0.0567 0.0567 224.667 0 0 MPI_Get_count() 0.0 0.037 0.037 1 0 37 MPI_Comm_free() 0.0 0.00867 0.00867 1 0 9 MPI_Errhandler_set() 0.0 0.008 0.008 2 0 4 MPI_Group_rank() 0.0 0.00667 0.00667 2 0 3 MPI_Comm_compare() 0.0 0.005 0.005 1.33333 0 4 MPI_Comm_group() 0.0 0.00267 0.00267 1.33333 0 2 MPI_Group_size() 0.0 0.00133 0.00133 0.333333 0 4 MPI_Group_translate_ranks()

...which I can't pretend to understand. Reasonably, the first line would be the cpu time and the wall time (4.8 and 7.6 s vs 5,271 and 8,643 ms).

A visual representation can be had by launching paraprof:
paraprof


Now it's time to explore...

The one thing that doesn't seem to work is visualisation of the communication matrix...



Failed attempt to build with vampirtrace
sudo mkdir /opt/tau
sudo chown $USER /opt/tau
cd /opt/tau


wget http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/software_werkzeuge_zur_unterstuetzung_von_programmierung_und_optimierung/otf/dateien/OTF-1.12.2salmon.tar.gz
tar xvf OTF-1.12.2salmon.tar.gz
cd /OTF-1.12.2salmon/
./configure --prefix=/opt/tau/OTF
make
make install
cd ../


wget http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/software_werkzeuge_zur_unterstuetzung_von_programmierung_und_optimierung/vampirtrace/dateien/VampirTrace-5.14.1.tar.gz
tar xvf VampirTrace-5.14.1.tar.gz
cd VampirTrace-5.14.1/
./configure --prefix=/opt/tau/vampirtrace --with-mpi-dir=/usr/lib/openmpi/lib --with-extern-otf-dir=/opt/tau/OTF
make
make install


wget http://tau.uoregon.edu/tau.tgz
tar xvf tau.tgz
cd tau-2.22-p1/
./configure -mpilib=/usr/lib/openmpi/lib -prefix=/opt/tau -openmp -TRACE -iowrapper -otf=/opt/tau/OTF -vampirtrace=/opt/tau/vampirtrace
make install

It builds fine, but during execution of mpirun -n 2 tau_exec... I get
Error: No matching binding for 'mpi' in directory /opt/tau/x86_64/lib
Available bindings (/opt/tau/x86_64/lib):
Error: No matching binding for 'mpi' in directory /opt/tau/x86_64/lib
Available bindings (/opt/tau/x86_64/lib):
  /opt/tau/x86_64/lib/shared-disable
  /opt/tau/x86_64/lib/shared-disable

311. Compiling MPE for MPI profiling

I've been wanting to get an overview of how my parallel (nwchem, gamess, dalton, etc.) programs are running, and whether there are any obvious bottlenecks other than the network and slow harddrives that I can sort out.

The Australia high performance computer facility in Canberrra uses http://ipm-hpc.sourceforge.net/, but I'm not having much luck compiling it, and the lack of recent updates makes me somewhat less willing to invest too much effort into it.

So I stumbled across MPE instead: http://www.mcs.anl.gov/research/projects/perfvis/download/index.htm#MPE

The problem is that almost all of the links of that page are broken, including those pointing towards the documentation, so I don't actually know how to use it properly. The presence of mpecc in /opt/mpe/bin suggests that it's used as a stand-in for mpicc when compiling, which I'll test some day.


Installing/compiling
cd ~/tmp
wget ftp://ftp.mcs.anl.gov/pub/mpi/mpe/mpe2.tar.gz
tar xvf mpe2.tar.gz
cd mpe2-1.3.0/
./configure MPI_CC=mpicc MPI_F77=mpif77 --prefix=/opt/mpe
make
sudo make install

I found what I looked for in Tau instead: 
http://verahill.blogspot.com.au/2013/01/312-tau-mpi-profiling-on-debian.html

310. Remote mounting using sshfs

I've run out of USB ports on my work desktop, so I occasionally cheat and attach USB drives to one of my compute nodes and transfer the files across the network to my desktop. Since I've got a gigabit switch set up, the speeds are quite acceptable.

NFS isn't really a solution here. Instead, sshfs is the tool to use.

The local and remote computer will be referring to Desktop and Node, respectively. The specific example I'm using here is that of a USB drive manually mounted on the Node, which contains pictures that I want to transfer to my Desktop.

On the Node
The plugged in USB device is found as /dev/sdb, and holds only one partition, /dev/sdb1.

sudo mkdir /media/usbdrive
sudo mount /dev/sdb1 /media/usbdrive


On the Desktop

sudo apt-get install sshfs
sudo mkdir /media/remote
sudo sshfs $USER@Node:/media/usbdrive /media/remote -o allow_other

That's about it. To unmount do
sudo umount /media/remote

and
sudo umount /media/usbdrive

respectively.

08 January 2013

309. Installing Office 2003 in Wine on Debian Testing

UPDATE 4 Feb 2013: Equation Editor under Office 2003 used to be support on Wine/Crossover. The devs are aware of it and looking into fixing the regression.


Bug reports to that effect:
http://bugs.winehq.org/show_bug.cgi?id=32697 and http://www.codeweavers.com/support/tickets/browse/?ticket_id=931759


UPDATE: I've since tried this using CrossOver 12 as well. The installation goes much more smoothly (the .exe file works fine), and you basically don't need to do anything except click 'Next' a couple of time. The Equation editor still doesn't work though. There's a free 15 day trial version of CrossOver available at http://www.codeweavers.com/products/. If running Office 2003 is your goal there seems to be little difference between CrossOver and Wine. However, I ended up buying a copy of crossover anyway since at this stage of my life and career it's not that much money and by supporting crossovre you indirectly support wine.

Since I would guess at the lack of Office for Linux as the single most problematic issue facing those wishing to switch from Windows or Mac to Linux, and since it would be something attempted very early on in someone's experience of Linux, it doesn't hurt to show the steps in detail -- especially since there are a few stumbling stones.

The linux community like to suggest new adopters to run Word under Wine, or better yet, to use libre/openoffice. It's an open secret that there are issues though. Most casual users won't notice most of the time,  but problems do exist.

In the particular case of Office 2003 under Wine, my main issue is that there's no way of installing Microsoft Equation Editor from what I can see, and it is a feature I do use when publishing together with other people (sweet, sweet TeX...). Another issue is that figures tend not to show up well or at all (complaints about memory -- most of our collaborative files are >5 Mb) when I use Word under Wine. These aren't issues when running in a virtual machine, which is the way I usually suffer through my bouts with Office.

An added head ache is this: http://wine.1045685.n5.nabble.com/Bug-26358-New-Office-2003-installer-crashes-immediately-td3414859.html and  http://bugs.winehq.org/attachment.cgi?id=35449. Using any of the versions of Wine above I just can't install using the setuppro.exe. This bug report ( http://bugs.winehq.org/show_bug.cgi?id=31942) showed me the simple solution though: using pro11.msi instead.

Anyway, just be aware of the limitations -- you can't install office, then do wine control, "add/remove programs", and add e.g. Microsoft Equation Editor. Likewise, if you try to be smart and do a custom install, and choose advanced customization, the installation will fail and complain about missing CAB files (in the case of equation editor: 01561403.cab). I've also tried installing MathType 4, which shows up under Objects, but is reported as missing when selected.

Otherwise this works fine.

Wine
To install the official wheezy version of wine:
sudp dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install wine wine-bin:i386 libwine-bin:i386 libwine:i386

You can also see the following posts on this blog for how to install/compile wine:
1.5.21 (using multiarch)
1.5.19 (using the dev.carbon-project.org rules)
1.5.5 (using the dev.carbon-project.org rules)
1.5.5 (downloading precompiled debs)

For this installation I've used Wine 1.5.19.

Installation
First set up cabextract and winetricks and install corefonts etc.
sudo apt-get install cabextract ttf-mscorefonts-installer
wget http://winetricks.org/winetricks
chmod +x winetricks
./winetricks corefonts fontfix

If you have Office 2003 as an OFFICE11.iso file (likely if you're at e.g. a university) then loop mount that file
In the directory where your iso file is:

sudo mkdir /media/OFFICE11
sudo mount -o loop OFFICE11 /media/OFFICE11
mount: warning: /media/OFFICE11/ seems to be mounted read-only.
wine msiexec /i /media/OFFICE11/pro11.msi

If you get any questions about Gecko or Mono, go ahead and let them install. Anyway, this launches the installation -- just do what you'd do under Windows, with the following caveats:

Important: You can't go back and install missing programs, so install all that you want from the beginning. Also, if you try to do advanced customisation and install Equation Editor (maybe other programs too) the installation will fail and complain about a missing CAB file. It hasn't always been so  apparently.


Running
Launch it as any other gnome or kde program, or from the terminal do
wine ~/.wine/drive_c/Program\ Files/Microsoft\ Office/OFFICE11/WINWORD.EXE

308. Compiling Wine 1.5.21 with multiarch on Debian Testing -- clean multiarch build

UPDATE 16 May 2013: See here for Wine 1.5.30: http://verahill.blogspot.com.au/2013/05/416-wine-1530-in-chroot.html

Another update (28/1/2013): It seems like your mileage in terms of how well Wine will work for you will vary using this method. It works fine for me and I suspect it's because I've built earlier Wine versions of my systems and have ia32-libs-dev installed. This package is no longer available in Testing. ia32-libs installs the libs you need, but does not provide symlinks between .so and .so.X files -- you can probably do that by hand if necessary though. It's not for beginners.

An additional thing to remember is that the 1.5 series of Wine is a development series -- 1.4 is the stable series which is meant for end users. Progress made in 1.5 will be included in 1.6.


Update2 (26 Jan 2013): there are a couple of issues, and they will depend on how your system is set up.
* libosmesa6-dev:i386 will require mesa-common-dev:i386 which will remove libosmesa6-dev:amd64 and more, which is bad.
* libglu1-mesa-dev:i386 depends on libgl1-mesa-dev:i386 which will remove libgl1-mesa-dev:amd64 and more, which is bad.
*  libgstreamer-plugins-base0.10-dev:i386 depends on libgstreamer0.10-dev:i386 which depends on gir1.2-gstreamer-0.10:i386 which depends on gir1.2-freedesktop:i386 which depends on gir1.2-glib-2.0:i386 which depends on libgirepository-1.0-1:i386 which will remove a whole lot of packages (132 on one of my systems, including gnome, gdm3 etc.)

I'm working on figuring out what's triggering this on some systems but not others. Be aware that that doesn't necessarily mean that I will be successful in doing so -- any help is appreciated.

In the mean time see http://verahill.blogspot.com.au/2013/01/308-compiling-wine-1521-on-debian.html to see how to build wine in a chroot, which is safe. You can then install that .deb package on your normal system and HOPEFULLY there won't be any broken dependencies. You won't need the -dev:i386 packages on the install target.


ldd `which wine` 
linux-gate.so.1 => (0x55573000) libwine.so.1 => /usr/local/lib/libwine.so.1 (0x55576000) libpthread.so.0 => /lib32/libpthread.so.0 (0x556ec000) libc.so.6 => /lib32/libc.so.6 (0x55705000) libdl.so.2 => /lib32/libdl.so.2 (0x55867000)
so you really don't seem to need much in the way of shared libs installed.



Old Update: This build is safe now, and there are no remaining errors (i.e. opencl has been taken care of) (see update2 above for more recent information)


I've been building wine (e.g. http://verahill.blogspot.com.au/2012/12/building-wine-1519-on-debian.html) on a system that was set up a few years ago.

Ergo, since ia32libs are on their way out and being replaced by multiarch libs it's time to have a look at what a build from scratch looks like, and what dependencies are needed.

This time we'll build wine the old-fashioned way using configure/make/make install. The biggest challenge was to overcome the annoying
configure: error: Cannot build a 32-bit program, you need to install 32-bit development libraries.
error. The solution to that was simply to install gcc-multilib. 

To work everything out from scratch I've set up a an amd64 chroot of a clean debian testing/wheezy. , It's increasingly appearing as if this is the easiest and safest thing to do.  Hopefully the release of Jessie will get the develop/improvement of multiarch back on the road.

To follow what's new in wine, look here: http://linux.softpedia.com/progChangelog/Wine-Changelog-148.html

...and the obligatory screenshot


The chroot environment
Basically as I've shown before:

sudo apt-get install debootstrap coreutils
mkdir -p $HOME/tmp/architectures/wine
cd $HOME/tmp/architectures
sudo debootstrap --arch amd64 testing $HOME/tmp/architectures/wine/ http://ftp.au.debian.org/debian/
sudo chroot wine/
apt-get install locales sudo vim
echo 'export LC_ALL="C"'>>/etc/bash.bashrc
echo 'export LANG="C"'>>/etc/bash.bashrc
source /etc/bash.bashrc
adduser winebuild
usermod -g sudo winebuild
echo 'Defaults !tty_tickets' >> /etc/sudoers

Edit /etc/hosts and add your hostname (here beryllium) to the end of the line e.g.
127.0.0.1 localhost beryllium

Exit, and the log in again
exit
sudo chroot wine

Then change user:
su winebuild
cd ~

Change at the very least the country code for the repo above (n red) so that you're using the closest/fastest one.

Compiling Wine
Set up multiarch (we're using amd64 but also want to support i386). I'm setting this up from scratch.

sudo dpkg --add-architecture i386
sudo apt-get update

Get the wine dependencies:

sudo apt-get install libx11-dev:i386 libfreetype6-dev:i386 libxcursor-dev:i386 libxi-dev:i386 libxxf86vm-dev:i386 libxrandr-dev:i386 libxinerama-dev:i386 libxcomposite-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libglu-dev:i386 libosmesa-dev:i386 libdbus-1-dev:i386 libgnutls-dev:i386 libncurses-dev:i386 libsane-dev:i386 libv4l-dev:i386 libgphoto2-2-dev:i386 liblcms-dev:i386 libgstreamer-plugins-base0.10-dev:i386 libcapi20-dev:i386 libcups2-dev:i386 libfontconfig-dev:i386 libgsm1-dev:i386 libtiff-dev:i386 libpng-dev:i386 libjpeg-dev:i386 libmpg123-dev:i386 libopenal-dev:i386 libldap-dev:i386 libxrender-dev:i386 libxml2-dev:i386 libxslt-dev:i386 libhal-dev:i386 libcurl4-openssl-dev:i386

sudo apt-get install gettext prelink bzip2 bison flex oss4-dev checkinstall ocl-icd-libopencl1:i386 opencl-headers

sudo apt-get install gcc-multilib

mkdir ~/tmp/
wget http://mirrors.ibiblio.org/wine/source/1.5/wine-1.5.21.tar.bz2 -O wine-unstable_1.5.21.orig.tar.bz2
tar xvf wine-unstable_1.5.21.orig.tar.bz2
cd wine-1.5.21/
./configure
config.status: executing include/wine commands config.status: executing Makefile commands configure: Finished. Do 'make' to compile Wine.
time make -j4

where -j4 is the number threads to launch, normally number of cores+1 (see here). Took around 15 minutes on a 3 core AMD.

Next
sudo checkinstall

to build a .deb package and install it (preferred solution) or
sudo make install

to just install.
Done!


Note that while this build works without issue, the transition to multiarch in debian is not complete and you will likely encounter this problem:
p11-kit: couldn't load module: /usr/lib/i386-linux-gnu/pkcs11/gnome-keyring-pkcs11.so: /usr/lib/i386-linux-gnu/pkcs11/gnome-keyring-pkcs11.so: cannot open shared object file: No such file or directory
If you try to fix it you get
sudo apt-get install gnome-keyring:i386 libcap-ng0:i386 libgck-1-0:i386 libgcr-3-1:i386 gcr:i386 libcap2-bin:i386
The following packages have unmet dependencies: libgcr-3-1:i386 : Depends: libgcr-3-common:i386 but it is not installable
This is known. The question is what the consequences of this are -- in this thread it's been suggested that it prevents any program needing to use SSL under wine from working. More at wine HQ here and here. More at Debian here. If I'm reading this right -- and I might not be -- we'll at the least have to wait for gnome-keyring 3.6 to make it out of experimental to sid, then from sid to testing (which is frozen).

 For no particular reason I think this may be what's causing issues with the installation of Office 2003 (and a few other programs, including Agilent HP Chemstation).

Links to this page:
http://linux.org.ru/forum/desktop/8571617/page1?lastmod=1358076186524

307. Burning audio CDs from the command line in debian testing/wheezy

I've got a CD burner on a headless box, so burning audio CDs from the command line is a necessity.

I also normally end up burning flash videos that I've converted to mp3s, so I'll show that too. This post is essentially covered (although not very well) already here: http://verahill.blogspot.com.au/2012/01/debian-testing-64-wheezy-small-fixes.html

First install the necessary programmes
sudo apt-get install ffmpeg wodim mpg123


Converting flv to mp3
To batch-convert flv files to mp3, do
ls *.flv|xargs -I {}  ffmpeg -i {} -ar 44100 -ab 160k -ac 2 {}.mp3


Preparing the files
Rename your files to 01.mp3, 02.mp3 etc. to make the songs burn in that order (since you're using *wav below).

Convert the mp3s to wav files (you could've gone straight from flv to wav above)
for i in *.mp3; do mpg123 --rate 44100 --stereo --buffer 3072 --resync -w "`basename "$i" .mp3`".wav "$i"; done

Burn
Burn with
wodim -v -pad speed=1 dev=/dev/cdrw1 -dao -swab *.wav

assuming that cdrw1 is the correct device.

Eject your cd when done.
eject cdrom1

Done.
[There's also no shortage of terminal music players, such as cplay.]

PS. You can burn anything you want from the command line using burn, e.g. an .iso file
sudo apt-get install burn
sudo burn -I -n myiso.iso

If the device you want to burn on is /dev/cdrom1 instead of /dev/cdrom, you can edit that in /etc/burn.conf

07 January 2013

306. Insync with Google Drive and Google Docs on Debian Testing/Wheezy

The problem:
1. It would be nice to be able to use Google Docs as a replacement for Microsoft Word until Libre/OpenOffice catch up (post about that later) or the world switches to LaTeX and
2. for that to happen there needs to be an easier way to sync documents between google docs and your harddrive than using email.

The closest thing to that is using Google Drive to keep documents synced, and opening them in Google Docs using your browser.

It's been more than half a year since promising that Google Drive would be available for linux, and Google has yet to actually release anything (here) and it almost looks like vaporware (here).

In lieu of an official solution, there are a few options. One is Grive -- which seems to work with Google Drive but not Google Docs -- and another one is Insync, which isn't open source as it is owned by a start-up. It's the most promising and full-featured solution though, so we'll go with that.


There used to be gdocsfs, but it doesn't seem to be maintained.

The usual caveats about installing things from outside the repos apply, and even more so in this case since the source code is not available.

Setting up Insync
sudo apt-get install xdotool python-nautilus libxdo2 gir1.2-nautilus-3.0
mkdir -p ~/tmp/insync
cd ~/tmp/insync
wget http://s.insynchq.com/builds/insync-beta-gnome-cinnamon-common_0.9.34_amd64.deb
wget http://s.insynchq.com/builds/insync-beta-gnome_0.9.34_all.deb
sudo dpkg -i *.deb

If all went well you'll find InSync installed (move mouse to top-right corner, type insync and it should be there). Clicking on it opens a browser tab, in which you're asked to select the gmail account you wish to use.

You're next asked to allow InSync to do various things:

Confirm (you'll then get an email) and associate your machine with the account.

You should now have a new set of folders in $HOME:
/home/me/Insync/
`-- me@gmail.com





If you create a directory either in ~/Insync/me@gmail or in google drive using your browser, the directory should show up in both places (i.e. it's synced) -- assuming that you've got insync running:
insync --headless > /dev/null &

will keep it running in the background. Any doc file copied to the insync folder will now be editable in Google docs by pointing your browser to https://drive.google.com/#my-drive


Simple as that.

06 January 2013

304. Getting started with Simon 0.4 on Debian Wheezy/Testing (very basic)

Here's how to get started with Simon 0.4 -- although be warned that I've never used Simon before, and that there are likely better resources out there.

In the few cases where I use the command line, I have presumed that you are using the same locations as shown in this post: http://verahill.blogspot.com.au/2013/01/303-building-simon-04-speech.html

In case you screw up, to wipe all previous settings, try:

find ~/.kde -name "simon*"|xargs -I {} rm {} -rf

If you're really desperate, nuke the entire ~/.kde folder, although that obviously has repercussions if you're actually using KDE and not GNOME. Also look under ~/tmp/$USER-kde -- I had simond put files there too.

Anyway.


Running simon
simond &
simon

If you need to kill simond you can do
kill %1

assuming that you don't have any other background procs in that terminal.

1. Scenarios:
Click on Open, select Download and pick the scenarios which you are interested in. To make sure that things are working I'm more or less following the video.
In Scenarios, select a couple of H4W scenarios  (e.g. keyboard, mouse etc.) -- BUT NOT THE FIREFOX ONE, which causes trouble.








2. Speech Models:
Click on Open model, select Download and pick the Speech Model which you want. Pick the  HUB4 model since from the Youtube video it appears that you should match your scenarios and Speech models e.g. VF with VF and H4W with HUB4.






3. Server
Nothing weird here:


4. Sound Devices
I run my stuff via pulseaudio -- i.e. whatever the input source is there, will be used.


5. Volume
Do a bit of talking and see how the volume pans out. Ideally you should have any amplification of your microphone turned off, since that causes higher noise levels.


Julius problem:
If after clicking Finish you see this, you will want to work out what went wrong:
Make sure that you
1. compiled Simon with -DPJULIUS correctly set, and
2. you export the directory:
export PATH=$PATH:$HOME/.simon/julius/bin

Then restart simond and simon e.g. quit simon and kill simond, then launch simond in the background and then simon. To test if it's working correctly, go to Actions/Synchronize. No error means that it's working.


6. Get dictionary
Do e.g.
cd ~/.simon/
mkdir files
cd files
wget http://www.repository.voxforge1.org/downloads/SpeechCorpus/Trunk/Lexicon/VoxForge.tgz
tar xvf VoxForge.tgz

then select e.g. Standard, and click on Open "Standard"



  Click on Import Dictionary:
Select Shadow Dictionary and Next:
Pick HTK lexicon:

Import your file:
You should now see it compile the model and there should be stuff under Shadow Dictionary:



Testing
At this point you should be able to test whether you're being recognised. Make sure that the button says 'Activated' and not 'Activate'. Try speaking a few commands e.g. 'One', 'Two', 'Three','Chaos','Control' etc.
'Chaos'

'Control'
You'll see if it works by looking at "Last recognition results:".


Thoughts
Stuff that doesn't work: Configure Audio from within simon, Configure Acoustic Model from within Simon. Adaptive training with the VoxForge base model (complaining about grammar)

Stuff that does work: speech recognition, adaptive training for the HUB4 base model (and beautifully so).

While it's easy enough to get started, it does appear that there's no easy way of running the wizard again in case you want to change e.g. the base model or install more scenarios via download.

Obviously such a piece of software is fairly complex, and will have a high error rate, yet this is the type of software which should ideally just sit in the background and not really be noticed by the user -- it should (ideally) not require more work than e.g. using a mouse or a keyboard.

We're not quite there yet -- hence why we're at 0.4 and 1.0. Anyway, it's still kind of neat when my garbled utterances are correctly (most of the time) recognised by Simon. Now, how to get it to actually do stuff for me?

A. Get Scenarios later
visit http://kde-files.org/index.php?xcontentmode=692&PHPSESSID=0e48f2edd26bf70e676459a5465ec675


B. Get Base models later
visit http://kde-files.org/index.php?xcontentmode=648

05 January 2013

303. Compiling Simon on Debian Testing (Simon 0.4)

Here's how to get Simon 0.4.0 compiled. Simon does speech recognition which allows you do voice-control your computer. It does NOT do transcription, e.g. like Dragon Naturally Speaking.

See e.g. here, here and here for more information about Simon.

Simon uses cmake. I hate cmake since it makes life a lot more difficult than it needs to be.

Also, note that Simon relies heavily on KDE, so you need a number of KDE related files -- if not the whole desktop -- installed. Obviously, it runs fine under GNOME, which is where I'm using it.

This post is limited to describing how to compile it, not how to use it -- that may come later.



First get the deendencies. The list of deps is taken from http://userbase.kde.org/Simon/Development_Environment#Requirements, and expanded (e.g. libboost).

sudo apt-get install qt4-qtconfig kdelibs5-dev libxtst-dev libsamplerate-dev kdepimlibs5-dev libboost-dev
sudo apt-get install build-essential cmake gettext kdeartwork libqwt-dev libqt4-sql-sqlite libphonon-dev libattica0 libattica-dev zlib1g-dev libasound2-dev 


Next sort out Julius
I think it used to be in the repos, but it's not there anymore from what I can see).

mkdir ~/tmp/simon -p
cd ~/tmp/simon
wget http://jaist.dl.sourceforge.jp/julius/56549/julius-4.2.2.tar.gz
tar xvf julius-4.2.2.tar.gz
cd julius-4.2.2/
./configure --prefix=~/.simon/julius
**************************************************************** Julius/Julian libsent library rev.4.2.2: - Audio I/O primary mic device API : alsa (Advanced Linux Sound Architecture) available mic device API : alsa oss supported audio format : RAW and WAV only NetAudio support : no - Language Modeling class N-gram support : yes - Libraries file decompression by : zlib library - Process management fork on adinnet input : no Note: compilation time flags are now stored in "libsent-config". If you link this library, please add output of "libsent-config --cflags" to CFLAGS and "libsent-config --libs" to LIBS. ****************************************************************
make make install cd ../


HTK
You'll need HTK to use adaptive models with the Voxforge base. It's not necessary otherwise.
To get HTK, register here: http://htk.eng.cam.ac.uk/register.shtml

You should receive a password immediately afterwards. Then go to http://htk.eng.cam.ac.uk/download.shtml and download the Linux sources.Put the HTK-3.4.1.tar.gz file in ~/tmp/simon.
cd ~/tmp/simon
tar xvf HTK-3.4.1.tar.gz
cd htk/
./configure --prefix=$HOME/.simon/htk --disable-hslab
make all
mkdir ~/.simon/htk
make install

For some reason it's built as 32 bit, not 64. Also, it wants the parent install directory to exist before running make install.

Next, sphinx:

wget http://sourceforge.net/projects/cmusphinx/files/sphinxbase/0.8/sphinxbase-0.8.tar.gz
tar xvf sphinxbase-0.8.tar.gz 
cd sphinxbase-0.8/
./configure --prefix=$HOME/.simon/sphinxbase
make
make install
cd ../


wget http://sourceforge.net/projects/cmusphinx/files/pocketsphinx/0.8/pocketsphinx-0.8.tar.gz
tar xvf pocketsphinx-0.8.tar.gz
cd pocketsphinx-0.8/
./configure --prefix=$HOME/.simon/pocketsphinx SphinxBase_CFLAGS=-I$HOME/.simon/sphinxbase/include/sphinxbase SphinxBase_LIBS=-L$HOME/.simon/sphinxbase/lib
make
make install
cd ../

wget http://sourceforge.net/projects/cmusphinx/files/sphinxtrain/1.0.8/sphinxtrain-1.0.8.tar.gz
tar xvf sphinxtrain-1.0.8.tar.gz
cd sphinxtrain-1.0.8/
./configure --prefix=$HOME/.simon/sphinxtrain SphinxBase_CFLAGS=-I$HOME/.simon/sphinxbase/include/sphinxbase SphinxBase_LIBS=-L$HOME/.simon/sphinxbase/lib
make
make install


I couldn't sort out OpenCV, so this Simon won't be watching you.


Finally, Simon.
wget http://mirrors.mit.edu/kde/stable/simon/0.4.0/src/simon-0.4.0.tar.bz2
tar xvf simon-0.4.0.tar.bz2
mkdir build/
cd build/
cmake -DCMAKE_INSTALL_PREFIX=~/.simon -DPOCKETSPHINX_LIBRARIES=$HOME/.simon/pocketsphinx/lib/libpocketsphinx.so -DPOCKETSPHINX_INCLUDE_DIR=$HOME/.simon/pocketsphinx/include/ -DSphinxBase_INCLUDE_DIR=$HOME/.simon/sphinxbase/include/ -DSphinxBase_LIBRARY=$HOME/.simon/sphinxbase/lib/libsphinxbase.so -DPJULIUS=$HOME/.simon/julius -DPSPHINX=$HOME/.simon/sphinxbase ../simon-0.4.0/

at which points you should see:
----------------------------------------------------------------------------- -- The following external packages were located on your system. -- This installation will have the extra features provided by these packages. ----------------------------------------------------------------------------- * LibSampleRate - Resampling library * KDE PIM Libs - KDE Libraries for PIM * Sphinxbase - Open source toolkit for speech recognition * PocketSphinx - PocketSphinx is a small-footprint continuous speech recognition system ----------------------------------------------------------------------------- -- The following OPTIONAL packages could NOT be located on your system. -- Consider installing them to enable more features from this software. ----------------------------------------------------------------------------- * qaccessibilityclient KDE client-side accessibility library Required to enable ATSPI plugin. * OpenCV OpenCV (Open Source Computer Vision) is a library of programming functions for real time computer vision Required for Simon Vision -----------------------------------------------------------------------------


And now a bit of a hackjob -- we give the sphinx include dirs above, and while it works fine for most include files (they use e.g. sphinxbase/includeme.h), a number of files (prim_type, sphnx_config.h etc.) point towards the wrong directory (e.g. #include ). Two solutions -- either edit the files or put symmlinks where the files are looking. Symmlinks work well for me.


ln -s ~/.simon/sphinxbase/include/sphinxbase/sphinx_config.h ~/.simon/sphinxbase/include/sphinx_config.h
ln -s ~/.simon/pocketsphinx/include/pocketsphinx/pocketsphinx_export.h ~/.simon/pocketsphinx/include/pocketsphinx_export.h
ln -s ~/.simon/pocketsphinx/include/pocketsphinx/cmdln_macro.h ~/.simon/pocketsphinx/include/cmdln_macro.h
ln -s ~/.simon/pocketsphinx/include/pocketsphinx/ps_lattice.h ~/.simon/pocketsphinx/include/ps_lattice.h
ln -s ~/.simon/pocketsphinx/include/pocketsphinx/ps_mllr.h ~/.simon/pocketsphinx/include/ps_mllr.h
ln -s ~/.simon/pocketsphinx/include/pocketsphinx/fsg_set.h ~/.simon/pocketsphinx/include/fsg_set.h

make
make install
cd ~/.simon/bin
export PATH=$PATH:/$HOME/.simon/julius/bin:$HOME/.simon/sphinxtrain/bin:$HOME/.simon/htk/bin:$HOME/.simon/bin
simond &
simon

I haven't really explored it yet, but here are the first few dialogues:




There's an official video which you may want to watch before you get started:



Issues:
Stuff that doesn't work: Configure Audio from within simon, Configure Acoustic Model from within Simon
Stuff that does work: speech recognition.

Note:
If you set it up 'wrong' the first time around (or it crashed coding samples as above)) and want the wizard to run again, delete all the configuration files:
rm ~/.kde/share/apps/simon* -rf
rm ~/.kde/share/config/simon*

Links to this post:
http://www.forum-raspberrypi.de/Thread-speech-to-text?pid=13465

23 December 2012

300. Briefly: Sharing a folder using SAMBA on Debian

I don't ever use samba, but it's not a bad thing to know how to set up in case you need to share files with someone using Windows in a pinch.

First install samba:
sudo apt-get install samba samba-common smbclient

To get a share up with samba, create an /etc/samba/smb.conf and stick the following in it:


[global]
workgroup=WORKGROUP
guest account=nobody
security=shared


[asharedfolder]
path=/home/lindqvist/shared
guest ok=yes
read only=no
writable=yes
browsable=yes
comment= SMB share

Restart samba:
sudo service samba restart

1. This is an insecure share i.e. <b>anyone can access it</> and edit everything.
2.. Also, by omitting "netbios name=  " you can use the IP address of the server as the hostname, but you could also specify e.g. "netbios name=niobium" and use that as the hostname in nautilus when you connect to the host server.

To set up a user- and password-based share, do



[global]
workgroup=WORKGROUP
security=user


[asharedfolder]
path=/home/lindqvist/shared
guest ok=no
read only=no
writable=yes
browsable=yes
comment= SMB share

You need to add and set the samba password, and enable the linux user you want to give access as well:
sudo smbpasswd -L -a -e lindqvist


There are a lot of other options that can be set. Two of the more interesting ones are probably

[asharedfolder]
createmask=0755
valid users=me myself irene

which means that any new files created in that share via samba gets chmod 755, and only the users me, myself and irene can connect.

But often a basic smb.conf is easier to manage and will do what you want it to.

22 December 2012

299. Briefly: Start autossh at boot on debian testing/wheezy

I have one of my work computers set up to create a reverse tunnel to my WRT54G router at home, so that by login in to my router at home and then connecting to localhost:19996 I can access my work network from home. The problem is that if I reboot my work computer remotely I need to make sure that it tries to recreate the reverse ssh tunnel.

The way to do that is by editing /etc/rc.local and putting the following in it:
su lindqvist -c 'autossh -N -f -M 29001 -R 19996:localhost:22 remoteuser@my.router.com' &

Make sure you put it before the line that says
exit 0

linqdvist is the user at on my work computer I want to be running the autossh, 19996 is the port I'll connect to on my home router to gain access to the ssh port on my work computer. Since I'm using WRT54G with Tomato the remoteuser is root, and I got my dns (remote hostname) as shown in this post: http://verahill.blogspot.com.au/2012/02/tomato-router-and-free-dns.html
It's that simple.

14 December 2012

291. Another nautilus-open-terminal related bug

We recently had issues with nautilus-open-terminal opening the terminal in $HOME no matter from which directory it was called: http://verahill.blogspot.com.au/2012/11/fix-for-nautilus-open-terminal-opens-in.html

In that post the fix was to do
gsettings set org.gnome.desktop.default-applications.terminal exec gnome-terminal

Which worked fine as long as the exec-arg remained '-x'. It seems to have changed to '-e' now, so during the past few days (since nautilus 3.4.2) 'open terminal here' doesn't open a terminal at all.

Diagnosis:
Looking at this: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=693894


gsettings get org.gnome.desktop.default-applications.terminal exec
'gnome-terminal'
gsettings get org.gnome.desktop.default-applications.terminal exec-arg
'-e'
Fix:
gsettings set org.gnome.desktop.default-applications.terminal exec x-terminal-emulator

It should now work normally.

29 November 2012

287. Compiling Wine 1.5.5 from source using the carbon-dev deb patches

UPDATE 16 May 2013: See here for Wine 1.5.30: http://verahill.blogspot.com.au/2013/05/416-wine-1530-in-chroot.html

UPDATE (10th Jan 2013): See here for Wine 1.5.21 using the multiarch approach: http://verahill.blogspot.com.au/2013/01/308-compiling-wine-1521-on-debian.html
. Use the updated build instead of what follows below.


Here's how to build the wine 1.5.5 packages without any errors.

I'm cheating and using the debian rules from dev.carbon-project.org.-- it's a small file and is a better solution than downloading the deb files from there.

Note that you will probably need more packages than the ones listed here.

1. Download the source and debian build patches

sudo apt-get install bzip2
cd ~/tmp
mkdir wine-1.5.5_carbon/
cd wine-1.5.5_carbon/
wget http://sourceforge.net/projects/wine/files/Source/wine-1.5.5.tar.bz2
mv wine-1.5.5.tar.bz2 wine-unstable_1.5.5.orig.tar.bz2
tar xvf wine-unstable_1.5.5.orig.tar.bz2
cd wine-1.5.5/
wget http://dev.carbon-project.org/debian/wine-unstable/wine-unstable_1.5.5-0.1.debian.tar.bz2
tar xvf wine-unstable_1.5.5-0.1.debian.tar.bz2
rm wine-unstable_1.5.5-0.1.debian.tar.bz2 

2.  Edit control, control.in and rules
Allow any version of gcc (this may obviously backfire if your version is no good)
Edit debian/control and control.in

 27  gcc-4.5-multilib [amd64 kfreebsd-amd64], gcc-4.5 [amd64 kfreebsd-amd64]
 28  g++-4.5 [amd64 kfreebsd-amd64],

so that is says
 27  gcc-multilib [amd64 kfreebsd-amd64], gcc [amd64 kfreebsd-amd64],
 28  g++ [amd64 kfreebsd-amd64],

Edit debian/rules
 53 ifeq ($(DEB_HOST_GNU_CPU),x86_64)
 54 CC=gcc-4.5
 55 CXX=g++-4.5
 56 else
 57 CC=gcc
 58 CXX=g++

to say
 53 ifeq ($(DEB_HOST_GNU_CPU),x86_64)
 54 CC=gcc
 55 CXX=g++
 56 else
 57 CC=gcc
 58 CXX=g++

2. Get dependencies
Note that this list is based on a very old post -- some packages may not be needed.


sudo apt-get install ia32-libs ia32-libs-dev bison flex gcc libc6-dev libfontconfig-dev libfreetype6-dev libglu-dev libgsm1-dev libice-dev libjpeg-dev libldap-dev libmpg123-dev libncurses5-dev libopenal-dev libpng-dev libsm-dev libssl-dev libusb-dev libx11-dev libxcomposite-dev libxcursor-dev libxext-dev libxi-dev libxinerama-dev libxml2-dev libxrandr-dev libxrender-dev libxslt-dev libxt-dev libxxf86vm-dev make libcapi20-dev liblcms-dev libsane-dev libhal-dev libdbus-1-dev valgrind prelink libcups2-dev opencl-dev lib32opencl1 oss4-dev gettext lib32v4l-dev lib32ncurses5-dev lib32asound2-dev libtiff4-dev libgphoto2-2-dev
sudo apt-get install libxkbfile-dev libxxf86dga-dev freeglut3-dev unixodbc-dev gcc-multilib dpkg-dev

Note: I used the following list on debian stable/squeeze.
sudo apt-get install ia32-libs ia32-libs-dev bison flex gcc libc6-dev libfontconfig-dev libfreetype6-dev libglu-dev libgsm1-dev libice-dev libjpeg-dev libldap-dev libmpg123-dev libncurses5-dev libopenal-dev libpng-dev libsm-dev libssl-dev libusb-dev libx11-dev libxcomposite-dev libxcursor-dev libxext-dev libxi-dev libxinerama-dev libxml2-dev libxrandr-dev libxrender-dev libxslt-dev libxt-dev libxxf86vm-dev make libcapi20-dev liblcms-dev libsane-dev libhal-dev libdbus-1-dev valgrind prelink libcups2-dev oss4-dev gettext lib32v4l-dev lib32ncurses5-dev lib32asound2-dev libtiff4-dev libgphoto2-2-dev libxkbfile-dev libxxf86dga-dev freeglut3-dev unixodbc-dev gcc-multilib dpkg-dev debhelper libxmu-dev dctrl-tools lzma sharutils lib32z1-dev libgstreamer-plugins-base0.10-dev
sudo apt-get install lib32opencl1


Then continue as normal.

If you're on Wheezy/Testing you should be following this post instead anyway.

3. Compile
in wine-1.5.5/:
dpkg-buildpackage -us -uc
If all went well you saw:
dpkg-deb: building package `wine-unstable' in `../wine-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `wine-bin-unstable' in `../wine-bin-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-dbg-unstable' in `../libwine-dbg-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-dev-unstable' in `../libwine-dev-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-unstable' in `../libwine-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-alsa-unstable' in `../libwine-alsa-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-bin-unstable' in `../libwine-bin-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-capi-unstable' in `../libwine-capi-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-cms-unstable' in `../libwine-cms-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-gl-unstable' in `../libwine-gl-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-gphoto2-unstable' in `../libwine-gphoto2-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-ldap-unstable' in `../libwine-ldap-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-openal-unstable' in `../libwine-openal-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-oss-unstable' in `../libwine-oss-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-print-unstable' in `../libwine-print-unstable_1.5.5-0.1_amd64.deb'.
dpkg-deb: building package `libwine-sane-unstable' in `../libwine-sane-unstable_1.5.5-0.1_amd64.deb'.
[..]
dpkg-source: info: using options from wine-1.5.5/debian/source/options: --compression=bzip2 --compression-level=9
dpkg-source: info: unapplying Bug#28201_proposed-fix-modified.patch
dpkg-source: info: unapplying Bug#28898_squashed-proposed-patches.patch
dpkg-source: info: unapplying fix-winegcc-paths.patch
dpkg-source: info: unapplying Bug#29669_proposed-fix.patch
dpkg-source: info: unapplying function_grep.patch
dpkg-source: info: unapplying readd_xpm.patch
dpkg-source: info: unapplying debian-changes-from-1.1.32-1.patch
dpkg-buildpackage: full upload (original source is included)

Your packages will be found in the wine-1.5.5_carbon/ directory.
../libwine-alsa-unstable_1.5.5-0.1_amd64.deb
../libwine-bin-unstable_1.5.5-0.1_amd64.deb
../libwine-capi-unstable_1.5.5-0.1_amd64.deb
../libwine-cms-unstable_1.5.5-0.1_amd64.deb
../libwine-dbg-unstable_1.5.5-0.1_amd64.deb
../libwine-dev-unstable_1.5.5-0.1_amd64.deb
../libwine-gl-unstable_1.5.5-0.1_amd64.deb
../libwine-gphoto2-unstable_1.5.5-0.1_amd64.deb
../libwine-ldap-unstable_1.5.5-0.1_amd64.deb
../libwine-openal-unstable_1.5.5-0.1_amd64.deb
../libwine-oss-unstable_1.5.5-0.1_amd64.deb
../libwine-print-unstable_1.5.5-0.1_amd64.deb
../libwine-sane-unstable_1.5.5-0.1_amd64.deb
../libwine-unstable_1.5.5-0.1_amd64.deb
../wine-bin-unstable_1.5.5-0.1_amd64.deb
../wine-unstable_1.5.5-0.1_amd64.deb

If something went wrong, it's probably because of a missing dependency.

To install just run
sudo dpkg -i *.deb in the wine-1.5.5_carbon/ directory.