Déjà vu

So Becky and I have been trying to go back to the ship for a while now.

Not the Doulos, but the Logos Hope.

Somehow, I just don’t seem to have got it out of our blood yet.

We though about going out late January time, perhaps flying with the new recruits from the January Conference.

So, the Déjà vu thing…

We spoke to some people in August at the GO conference, sent some emails around, waited a bit.

Waited some more.  As one does.

Anyway, we got an email last week from our friend Mark, who’s one of the big manager types.

“I’ve spoken to the guys on board. I don’t suppose you could come at the end of the month, could you? I know it’s a bit short notice…”

!!

(Those previous two exclamation points without corresponding lexical complication are in fact, grammatically correct. ‘Self-explanatory punctuation’ I call it.)

 So it looks like we may be back on board in 3 weeks!  Quite exciting, and a perfect excuse to start up this blog again.

Why the ‘deja vu’ comment? Well, it seems awfully reminiscent of my first short term ship back in 2005.  Feel free to find the archive entries if you wish!

The Logos Hope is just coming out of a very long (9 months odd) layup/drydock, and so they’d really like a couple of extra pairs of hands to help in the programmes/events/Audio-Visual stuff sooner rather than later.

More details to come later. 🙂

Duel Booting, mapping UIDs for different filesystems

I’m dual booting Debian & OSX on my work machine.

In OSX, users are created starting with UIDs 1000.  So the admin user is 502, I’m 501, and so on.  Also, the ‘staff’ group ID is 20.

On Debian, users are created starting with UIDs 1000.  So I’m 1000, my testing blank account is 1001, and so on.  The ‘staff’ group ID is 50.

This is a bit of a problem when I want to share files.  Since it’s just me on the computer,
technically I could just mount the drive with full read/write permission to me.  But I don’t want that.  I don’t want to be able to muck around with stuff outside my /home (or /Users/) directory without really trying hard.

The way to do it is using FUSE and bindfs.

Debian squeeze comes with FUSE installed, and has bindfs, but the version of bindfs that comes in the repo wasn’t up to date enough, so you may need to compile your own.

sudo aptitude install bindfs libfuse-dev

 Download, compile and install a new bindfs:

http://code.google.com/p/bindfs/

tar -zxvf whicheverfileversion.tar.gz
cd whicheverfileversion
./configure
make
sudo make install


Should now leave you with an up to date /usr/local/bin/bindfs
  
For the sake of example, I’ll show my set up.  /media/OSX is the original.

sudo /usr/local/bin/bindfs –map=501/1000:@20/@50 OSX OSX

Sweet! It works!

There will be quite a big performance hit from using bindfs.  Some benchmarks (linked from the bindfs page) reckon about 80%.  Not a problem for me here, since it’s fast enough.  But to be aware of.

Here’s a script which I put in /etc/init.d/osxfixperms.sh to run this on bootup.

#! /bin/sh
### BEGIN INIT INFO
# Provides:          osxfixperms 
# Required-Start:    fuse
# Required-Stop:
# Default-Start:     S
# Default-Stop:      0 6
# Short-Description: remount OSX drive with correct permissions.
# Description:
### END INIT INFO
do_start() {
    /usr/local/bin/bindfs –map=501/1000:@20/@50 /media/OSX /media/OSX
}
case “$1” in
  start|””)
    do_start
    ;;
  restart|reload|force-reload)
    echo “Error: argument ‘$1’ not supported” >&2
    exit 3
    ;;
  stop)
    umount /media/OSX
    ;;
  *)
    echo “Usage: osxfixperms.sh [start|stop]” >&2
    exit 3
    ;;
esac
:

Which I’ve then installed (on debian) with (as root)

update-rc.d osxfixperms.sh defaults

You’ll need to be mounting the OSX drive *BEFORE* this happens.  Generally, that means stuffing it in /etc/fstab in the appropriate way.  You can ask google / debian docs.  It’s fairly well documented.  Unlike using bindfs.

There.
Easy. (ish).

Python string concatination speed

I made a quick python script to convert OpenLP song (lyric) databases into the presentation format used by ProPresenter.

https://github.com/danthedeckie/OpenLP-To-ProPresenter5-Converter

is the link.  I put it together in a few hours, it should have been quicker, but I’m still re-acquainting myself with python.

It is a nice language.

One thing I did today, while cleaning up a bit, was wonder about something I remember from doing python years ago – string concatenation.  Joining two strings (texts) together.

“Hello” + “World” -> “HelloWorld”

I remembered something about it being slow, and python recommending using

”.join((“Hello”, “World”))

which seems to me one of the most blatently ugly obscure gotchas I’ve come across in a long while.

Anyway, I refactored my code into that style – converting databases and song lyrics and writing XML stuff is pretty much all Text formatting and concatination.

It made no discernable difference.  So I went back to normal easier to read x += y, and x = y + z type of code.

2 weeks in Cyprus

So Becky and I came over to Cyprus for a couple of weeks.  The last time we visited was almost 2 years ago, so this is our first time here as a married couple.

It’s relaxing, and interesting, and fun, and weird being back in Cyprus.  A few things have changed, many people come and go, which is usual…

We’ve been catching up with loads of people from the local church (various congregations), been sailing with my dad twice, been to a performance at theatre antidote, looked at their lighting system and helped look at the audio stuff they have, played loads of Settlers of Catan & Ticket to Ride with my parents and their usual Catan-Cronies (hi guys! Hope you like your new title…), we visited Nicosia, and went across (briefly) to the North, spent time with the cats (me being somewhat cat-starved), went for walks by the salt-lake, I bought a Cyprus coffee maker, and some other bits of (tasteful) Cypriotness.  Partly because I like Cyprus coffee, and partly because I hope we’ll be heading back towards the ship(s) at some point, and would like to have some stuff from the country to show people.

I can’t figure out how to make the webcam take pictures (I’m writing on mum’s chromebook) so you’ll just have to wait for photos of the coffee maker.

Anyway. (I can’t write a blogpost without at least one paragraph starting with ‘Anyway’.) We’re off to the UK again in a couple days, which will be odd again.

As I mentioned, briefly, we are looking at heading back towards the ship(s) at some point.  Probably for a few months at the end of the year, if we can, and see if maybe we should be going out there for a few years again at some point – when, we don’t know.

I’m looking forward to seeing our friends in the UK again soon,  but not really looking forward to the weather, or work, to be honest.  Seeing live theatre stuff again… I miss it a lot.  And being at sea, even if it is on a little boat.  And warm weather, even if not that warm.    OMNIvision is doing some cool stuff, and the team is amazing, and I’m glad to be part of it.  But documentary-type video production, media-engineering and web development are kind of interesting, but not really exciting any more for me.

Thinking that next week I’ll probably be fixing broken mic cables and fighting stupid Internet Explorer broken-ness makes me very gloomy.

Still.  I’m in Cyprus now, spent the day sailing with my wife, my dad, currently I have a cat purring on my lap, about to play a game of Settlers, and go out for a meal this evening with some old friends, so, I really shouldn’t complain!  This has been a really good fortnight.

Techy post

I appologise for how techy and geeky and boring this post is.

I even find it so myself.

But I need to get this off my chest, so I can actually write about interesting stuff later…

And some geeky people may actually find something comprehensible in here.  Perhaps even interesting.

If you want to read it, then if your eyes glaze over with techiness, then stop reading, drink a glass of water, scroll down to ‘And another thing‘, and read from there.

So then.

Most of my recent work has been on the omnitube.org website.

It’s a site I put together with David when I visited OMNIvision 4 years ago (now that seems weird!).  We put it together in about 2 weeks, based on a well known large complex CMS (Content Management System) called Joomla, which was the OM standard web-site-software-package, a few very large complex modules for that, specifically “VirtueMart”, plus a bunch of my own hacks in php/javascript, plus a load of very clever HTML/CSS from David.  Since then, I’d done a few fixes for them, logging on from a ‘net cafe in Singapore, and when Becky and I arrived at OMNIvision 2 years ago, it was basically in the same state.

The trouble was that in the 2 years since we’d put it together, the internet had changed quite a lot, and we hadn’t had the time to really work on OMNItube to change with it.  Joomla had advanced significantly, as had Virtuemart, and alas, we couldn’t smoothly upgrade to the latest versions of those without majorly breaking everything.

About a year ago (from now), I decided it was about time to do something about this, as I was scared that a now 3 year old install of joomla/etc without being able to have security updates was a bit scary.  Especially as we wanted to sell videos online, and DVDs, and so were doing credit-card type security stuff.  And the system admin wanted to upgrade the server, and I thought doing that would probably break everything again.

I spent about a week trying to upgrade things to the latest versions of joomla and virtuemart.  Some bits I got almost close to working, but in the end it just totally bogged me down.  Joomla + Virtuemart + all the plugins and modules which were installed to make everything work as we wanted ended up at around 7000+ files, and 800’000 lines of code.  Not joking.  So tracking down bugs and errors and trying even to find which function I was looking for took forever, and I simply don’t have a big enough brain to hold that much in my head at once.

So after a week or two of working hard to try and upgrade things, I gave up, it was just too big a job for me to do on my own.  I’d made no progress, and it was still a big stinkin’ mess.

We also had some new features we wanted to get working. Specifically, iTunes subscriptions, XML sitemap, facebook commenting/liking/etc, and related product lists.  The “search” system was also very broken, giving pretty much useless results most of the time.

I’d just read an article online about making stuff simpler, ‘the no-framework framework’ or something like that.  And looking at the SQL database, I figured I could write the iTunes and sitemap features we wanted in just pure basic php.  So I did.  It took about 3 days, which was pretty good, I thought, considering I’d not really done significant amounts of programming in about 4 years.

So then – perhaps alas? – I thought, well, why not re-write the whole of OMNItube in a similar manner?  I mean, how hard can it be?  It’s a very simple website, after all….

In 2 weeks after that, I’d put together a quick ‘demo’ of the site, which took it’s data from the joomla database, had all the basic functionality of the front end,  but in 700 lines of code, TOTAL.  No Joomla, No Virtuemart, nothing.

OK, so it was only the front end, if you wanted to actually add a new video or something you needed to edit the database manually, rather than use the horrendously over-complex somewhat-broken-by-our-CSS-mods joomla back-end, but hey!

700 lines of code is something I can hold in my head, no problem.  Way better than 800 thousand!

So anyway, that’s been the big project for me, over the last year.  It’s taken me 3 re-writes, which is what took so long.  I really should have spent a lot longer learning stuff and looking at alternatives before launching in to it as I did…

I first wrote it doing all the SQL in-line, sort of saying “I need a list of the products, so the SQL for that is SELECT…blah.blah.blah” and so writing it then and there.  Which works for very simple stuff.  But I began to find patterns, of course, and so wrote functions and classes to abstract stuff out, which was good!  But each time I did something like that, it meant re-factoring and writing stuff I’d already done, to make use of the new abstract methods I was coming up with.

Eventually, though, it was all working, pretty much.

 I slowly switched over bits of the live system to my new creation over time,  so the video embedding system was first, then the iTunes feeds, then the front end, eventually, and finally I was able to delete the joomla system, and run entirely on my much smaller simpler pure-php non-frameworked version.  I guess about 6 to 8 months ago?

So after the crazy summer, I did a bit of cleaning up, backups, version-management, etc, And a guy called Chris came to work with us, who is interested in doing web / programming stuff.   So I chatted with him, showed him OMNItube, as it was then, and told him about my future plans.

I’d found a cool looking database library for PHP called ‘Propel‘, which did much of the grunt work of doing SQL database querying instead of us having to do it.  I thought it looked cool, and would save us a lot of work in the future, and so Chris started work on porting OMNItube over to using Propel.  He also figured out a bunch of stupid stuff I’d been doing before, and found much better, simpler ways to do it.  Templating things, for the big example.

In some ways, I’d not really wanted to be working on any more OMNItube stuff for a while though.. I kind of thought ‘it’s working, I should probably just leave it until next year or something’.  But then after Chris left, I wanted to get his project all merged in to the main system, and not leave things in a mixed state, with 2 versions, his almost-done version, and the online-one with changes to it that I’d needed to make since he started….

Anyway.  I’ve spent the last few weeks hacking hard at OMNItube, until now it’s stable and running well using the new Propel based system.  It’s much simpler, much better, the administration/back-end thing is very cool, and generally just works great.

The whole thing is now 7000 lines of PHP, plus a few hundred of javascript and html, but still, that’s quite manageable, and I’m reasonably happy with it.  I’ve even open-sourced one of the smaller more generic bits of the system, and plan to put a few more modules online too, in the hope other people can find them useful in the future.

I’ve learned a lot, during this project.  I understand from experience a load of programming concepts a lot better than I used to before.  I’ve learned quite how much I dislike PHP, for one thing.  It really is a botch on top of a hack on top of a temporary way to fix a problem.  If I were starting the project now, I think I would pick a cleaner, simpler, better language.  Possibly ruby, python, LISP, javascript, perl5, or even haskell.

Another thing though, I’m really bored of web development.  It’s so much effort, and in the end, all you’re left with is a bunch of patterns of pixels on a screen.

I SO miss acting, theatre, drama, etc.

And the trouble is, after a day fighting with programming problems, I don’t then feel capable of doing anything creative.  It’s like my brain has just given up, as has my emotional capacity to care about creativity.  All I want to do is crash, which isn’t very nice for Becky, nor very helpful in building relationships, in staying up to date with anything actually interesting, or in keeping in practice with Clarinet or juggling or writing plays or whatever else I’d prefer, if I were less frustrated and computered-out to actually do.

Writing, for instance.  When I’ve been writing code, or trying to write code all day, I have no ability to write on this blog, say.  Or write newletters, or emails, letters, or even SMSs.

I’ve not really touched OMNItube now in about a week, which is why I’m able to write this post… but even still, I’m not quite able to break my brain out of that mould enough to write about anything interesting… all I can do is write about writing code.

Which I guess is a step in the right direction.

I’ll write a more general, non-techy post soon

Moving Users between computers on OSX

So we needed to move a bunch of users from an OSX Leopard Server to a normal OSX machine (as a backup / test bed)

using the normal ‘migration assistant’ didn’t work, so we had to do it manually. Possibly due to moving from a Intel based OSX-Server machine to a PPC based non-Server?

All user details are kept in

/private/var/db/dslocal/nodes/Default/users

in .plist files (XML, human readable).

and all passwords are kept in hash files in

/private/var/db/shadow/hash

The hash files are named after the “generateduid” found inside the users .plist file.

If you’re using home directories on the computer, you’d need to move them across too.

Ditto for user groups.  User groups are kept in

/private/var/db/dslocal/nodes/Default/groups

Move the files across to the new machine, and it all just works.

if you use “cp -n” on the command line, it won’t write over the top of files already there, so shouldn’t muck anything up (hopefully!)

NOTE:  One weird thing was that a bunch of the users didn’t appear in the Accounts Preference Panel.  This seems to be linked whether or not the account .plist file has a default shell set up:

    <key>shell</key>
    <array>
        <string>/bin/bash</string>
    </array>

it goes after the “realname” key/array pair.

There is a reason…

There is a reason why I prefer text based / commandline / unix interfaces.

“Why?” You may ask?

So I had to update a website – basically changing from ‘2011’ to ‘2012’ – for someone.  Once I got in to the file manager (web based) I could find the graphics files to replace, reasonably easily, and upload new ones.

But I needed to update the text of the banner.  No .html files, or .inc or anything to edit.

So I went to the “WYSIWYG” editor section of the site.  I could edit everything, except that part.  And the footer.  And the page title. So, I could edit the “articles”, but not much else.

After hunting around for a while, I decided to try the “Website Management” area.  Total different interface again, loads of settings boxes and tabs.  I found there on different pages where I could set the page title, and so on.  But I couldn’t find the banner! Anywhere!

Eventually, this morning, I found it.

In “Website Management” -> “Structure” -> “Website Settings” -> “Code Injection” -> “Advanced” -> “Site Banner HTML Override — Edited”

I hadn’t found it before, because the section I needed was “Site Banner HTML Override”, and to find that, I needed to look through a drop-down box which was displaying “Extra Header Code (within )” with some javascript includes.

SERIOUSLY.

What is wrong with just having a bunch of text files, and then letting me type

“grep 2011 *”

And it giving me a list of files to edit?  HOW is this graphical interface any easier?

OK.  So ‘grep’ isn’t common English, I know.  Learning basic unix does have a steep looking learning curve.  At first.

In case you’re interested, I started, months and months ago, writing an “English-like” command shell.

So you could type commands like:

“list all files which contain 2011”

and it should tell you.

It’s on sourceforge, the first alpha versions of the code.


http://sourceforge.net/projects/daftshell/

The whole idea of ‘natural language’ interfaces fascinates me.  I should work on daftshell again, one of these days.

So. Yeah.  Graphical “easy” systems aren’t always easy.

Windows Movie Maker

I wrote this a few months ago in a prayer meeting.  I know.  Very holy of me.

Windows Movie Maker [C:Program FilesMovie MakerMOVIEMK.exe] is a highly virulent productivity virus.  Not only effecting the computer it is hosted on, but having a socially engineered payload which self-propagates through USB sticks, video/data projectors, and various presentation media.

Reports have come in of this virus’ unique capability to be transmitted through analogue forms such as VHS tape, and occasionally film.

Symptoms:

  • Audience hypnotism
  • Hours of time wasted by ‘editor’ and ‘viewers’
  • Computer lock-ups and data-loss
  • Lowered expectations of quality
  • Copyright infringement
  • Physical injury
  • Death

Response Actions:

  • CTRL-ALT-DEL

[small print.  Dear lawyers, et al.  This is what’s known as parody.  Just like the word ‘Movie’ is in the program title.  In other words, a joke.]