previously known as [cold][wet][durham], [dirty][grimy][london],[busy][shiny][toon],[frantic][crowded][south]
Tonight I finally got all my favorite technologies talking to one another and now when i wave my wiiMote about in the air, little lights in my room get brighter or dimmer (and when I work out how to multiplex the serial signals you’ll be able to change which lights are on so change the colour!).
When I get a chance I’ll write it up.
So today another good idea got made into something. With everything ever for sale being available on the web, the price wars were always going to happen, and in some ways we’ve moved past them (I’ll buy from Amazon because it’s easy even if Jeff’s Mega Book World is a pound cheaper). Once you go beyond the rock bottom prices, the more interesting things include sales, and offers. I linked to a paper which found bargins on Amazon a while ago and today (via the magic of Twitter’s public timeline) I found Buy it Later, a Firefox plugin which lets you maintain a hotlist of stuff you’re interested in, and then messages you (emails or tweets) when they drop in price, or come back into stock.
I might sign up. I should sign up. I should get out of the habbit of buying books one at a time as I hear abuot them, and wang them all on a list like this instead.
My brain (and my rss feeds ) has been going into overdrive today.
Managing your time thought your calendar, diffiuclt to switch over to one imagines. Only works if people don’t expect instant responces to emails.
An interesting paradigm shift, not having a bad table, rather than having people thing ill of you (and websites having no excuse for the “bad table”).
Building an entire work life by dealing with crisises could be highly profitable, but might well not be doing what youy want or taking you where you want? If you can keep on top of the stress levels, there’s a lot of jobs to be had, but are the rewards for being the person who sorts out the root cause of the fires bigger? Maybe.
Hyperconnectedness V’s The danger of easy access, are these two sides of the same coin? Maybe twitters one-way-ness is the only reason it’s not a complete killer (and hasn’t caught on the way IM has to run large projects). Other people seem to be having the same thoughts.
The more I hear about muxtapes, the more I hear about muxtapes. muxtapes is everywhere. Internet makes tipping easier, if the product is good, and this reduces the need for advertising, which is can be evil. Speaking of evil marketting, I hovered over a targetted advert today on a site of which I am a memeber and noticed that embedded in the url was my name, my age, my sex, the last thing I searched for and the site this advert was being placed on. I’m sure thats breaking UK data protection acts, be aside, it gave me an evil idea.
A simple shell script calling curl repeatedly, and when they analyse their stats, they’ll find lots of people looking for unrelated subject (mostly very rude) with very silly names (all very rude) clicking on their adverts on some very rude sites (where they’re probably hoping their adverts are being placed). Karma?
I bought a wiiMote a little while ago because, well, they looked fun. No other real reason. Here’s how I got it up and running with my favorite programming environment, processing.org
Firstly, you need to make your laptop have “blue teeth” luckily my iBook came with these coloured nashers, which I’ve not really used and now wouldn’t be without. The wiiMote talks via these blue teeth to the iBook, so long as it has something to connect to.
Step in Darwiin, which is a wiiMote client. Install it, turn on the teeth, and press 1 and 2 togehter on the wiiMote and it just connects. Wave the wiiMote around in the air and you’ll see the wavey coloured lines representing it’s 3D velocity. It just works out of the box, which is nice.
There is a Darwiin version which rolls in OSC. OpenSoundControl is a protocol for communicating with devices. One nice feature is communication via TCP/IP, so a device can be connected to one machine and controlling another.
This also means that if you get a library that talks OSC, it can communicate with devices it cannot natively speak to, and such is the way with processing. Get the OSCP5 library and processing can now commuicate with Darwiin.
So far so good. Add in the WiiController class (included in the oscp5 download), and you’re ready to rock (and roll, lets not forget pitch control). Now you can access what the wiiMote is doing simply with wiiController.x, wiiController.y, and wiiController.pitch.
This was a good start, but moving a dot around the screen wasn’t that exciting. Poing however, now there is an adrenaline rush. So, crack open the collision detection, switch all the mouseY’s for wiiController.pitch (with a scaling factor) and bobs you uncle, you have wiiPong.
Now I’ve got myself into a huff about nation id cards before now, but it would appear that the government is steaming ahead, regardless as to whether people want them, re interested in them, or would pay for them. I listened to a discussion on Radio4 as I drove into work this morning with an MP advocating them but saying that “people wouldn’t adopt them unless they were free and actually had a use”, which made a lot of sense. There would appear to be two ways to go here, either they are completely free and you have to carry them (a la ID cards in many other countries, and ID cards during the war) or you charge for them but they have a major benefit (like a passport or a driving license). I prefer the latter, but any ID system where your populous doesn’t have to take part is never going to work (NI numbers being voluntary - never going to happen) so unless the card made thing substantially easier, faster or cheaper, it’s probably a no go.
Anyway, what I wanted to bring up wasn’t anything to do with the politics or economics of these cards, its their development methods. What will happen (I guarantee it) is that a couple of large IT provider will be asked to bid for the work, and it will narrow down to two, the cheap one which should be ok and the expensive one which will be rock solid, and the contract will got to the cheap one who will then try and bolt on some of the expensive system.
The key to an ID card, like any security pass, is that it should be extremely difficult to copy, clone or modify. If somebody can know their own up, or change theirs so when it’s scanned it doesn’t show them up as Jeff Smith the bank robber, then they become virtually pointless. This presents many challenges, after all the process for making a fake passport is well documented, and the actual leather folders and paper seem to be being stolen left right and centre. I watched a reported filming one being made on the BBC a few year ago, and the guy started by opening up a big box of them and picking one at random. They are clearly not kept well under wraps.
I imagine what will happen is that the cards method of storing information and revealing it will be shrouded in mystery, with big words like “unbreakable encryption” being banded around. It won’t be unbreakable, nothing is given enough time, power and money, but it should at least be something other than a bit of bit flipping. There’s lot of good encryption technology out there already, and much of it would be applicable.
Technology aside, I have a development idea - why not open source the entire thing? And don’t mass produce version one, or three, or even version 47. How up with a hard and narrow set of requirements eg. Must store full name, birthday, eye colour and a lookup_id (for comparing against a list). Something super simple. Then get a first version out there, and offer up £10,000 to every person who can find a crack, a hole, a work round or a sneaky way of misusing or modifying the data. Get 10,000 cards out there to ever academic, highschool kid, sneak bastard and clever git. If they find a hole, just pay up. Don’t quibble, try and say it’s not a hole, just bit the bullet, hand over the cash, and fix it.
Once you’ve had the first 100 bugs (and there will be that many), issue a new card, and the same challenge. Repeat as required. Set up a fast feedback loop and make it worth peoples time to break the damned thing. A sharp team will find 4 or 5 good holes in the first week, and they can go buy a new car. They’re happy, and you’ve for a more secure card. It won’t take more than 8 or 10 generations of these cards before something really bomb proof starts to show.
The other thing about being completely open about your spec and technology stack is that people will have a much better idea of the card and what’s on it. At the moment, most people are pretty suspicious of everything around ID cards, but if you get it all out there people can go and understand everything involved. It won’t quash everything, but it might help.
Security through obsfucated details is no security at all.
Looking though the logs for this place tonight I noticed how many people are going to old URLs which haven’t been active for months, years in some cases. 500 people a month were accessing www.coldclimate.co.uk/blog, which I retired about a year ago. Another 500 people were munching down on 2 of my homebrewed feeds www.coldclimate.co.uk/1.xml and /blog/feed.xml, neither of which have been around for about a year minmum. Throw in another few links and suddenly 1000 more pages are actually being served each month.
So I’ve added a couple of lines to my htacess file to redirect people. It’s dead simple really, add a line like “Redirect oldplace http://wwww.newplace.com/new” and any traffic that comes to /oldplace gets redirected. You have to fully qualify the new place however. Also, you htaccess file is parsed top down, you redirect /oldplace http://www.newplace.com and then in the line below redirect oldplace/really_old http://www.someotherplace.com then any traffic for oldplace/really_old will actually end up at http://www.newplace.com, not www.someotherplace.com.
As that explaination probably made as much sense as Northern Rock shares, I’d recommend this comprehencive yet easy to understand guide to htaccess. Also rememeber that htaccess can really bugger you up too
Lord knows who was visiting these long dead pages, and I suspect most of it is feed readers blindly grabbing my 404 page and not realising it, but hey, if I’ve suddenly reappeared on your radar, I’ve not been gone, honest. You may thank .htaccess’s wonderful Redirect command.
I spent ages (nearly two years) researching boids, flocking and autonomous character programming, but I still find articles such as this one about giving each member of a digital crowd its own personality could make animated mob scenes more realistic facinating. I’m sur e the name Demetri Terzopoulos is familiar too.
In the last week I’ve come across some brilliant tools to make my digital life easier, so I throught I’d better share. The Refresh Newcastle group put me onto some, and Google did the rest as and when I found I needed something.
MAMP
I do most of my development work on a php/mySQL stack, and normally I do it here on coldclimate, which is all very well and good, but it has all the problems of working remotely. Latency, need to be online, slow debugging, the potencial for a rogue piece of sql to drop my comments table etc, so some local development area was needed. My iBook has a local copy of Apache, and PHP installed, but installing mySQL was a ball ache, and upgrading to php5 looked like a nightmare. Step in MAMP, and all in one install which is easy as pie to get going with, but where each item in your tech stack is separate, meaning it’s much much easier to install upgrades, downgrades or even have multiple installs at the same time. Out of the box, it works a treat. (Mac only kids)
CodeIgniter
Following on from discussions about Ajax development kits, I was pointed to CodeIgniter, which is a php framework. MVC based (as all good code is in my book), I was knocking up a blogging site from scratch inside an hour as a play thing. Impressive, and means you can stop rolling all your own libraries. (Cross Platform - php based)
Musorg
Having set about getting my music collection under control I needed a method for batch editing mp3’s ID3 tags. Under Windows I use Tag and Rename, and it’s ok, but slow. On my Mac I’d just use iTunes, except then I’d need to import all my stuff into iTunes etc. ball ache city. Step in Musorg, which lets you pick I directory, and easily edit all your tags in one go, or file by file, quickly and easily. Remember to hit save before moving directory though! Once all the tags are sorted, I can throw the files as the Synology Cube, and forget about them. (Mac only)
jQuery
Another framework, this time proving aJax functionality quickly and fairly easily. I’m yet to really get going with this yet, but I’m hoping to use it and CodeIgniter to get my next project up and running quickly. (Cross platform - browser based)
coComment
Another handy tool born out of dicussions on the Refresh mailing list. I woke up with an idea which I hadn’t the inclination nor talent to write, so posted it up to see if anybody fancied a go at it. Somebody pointed out it existed already (always a good sign). coComment is a Firefox plugin which keeps track of all the comment you post on blogs, so you don’t loose track of where you said something and makes finding peoples replies easier. Now coComment+Google reader +rss = perfection. (webbased)
Sharepoint
Yes, I hate virtually everything about Sharepoint’s ideology and methodology, but once I got down off my high horse and stopped trying to have full control of the code it generated, I was able to get so much more done. Much as I don’t like working in it, if you use it in a completely point and click manner (even abstaining for FrontPage if possible) with some good templates, somebody with no technical experience can have a website with document stores, chat rooms, message posting, blogs and a wiki up and running in a few hours, which is quite incredible really. Yes, the code it generates is vile, portable as a breezeblock, fugly as mutant pug dog and only works on IE with a fixed screen resolution, but in a corporate environment with browser monoculture and fixed machien builds, this doens’t matter. It just works. Scary. Getting off my high horse was like biting off my own tongue though. (Microsoft only)
On my Windows boxes I install PDFCreator, which is generally very good. Tonight I went looking for an equivalent for Macs and luckily ran across a screen grab showing that… it does it natively! In any program which can print (and I only need to make flat PDFs, nothing exciting like links or embedded programming), you can simply pick “PDF” from a wee drop down, and then select “Save as…”. It even picks up the document title for the file name, so saving webpages for learning offline on the train is dead easy.
I love my iBook.
I love bashing Windows, but having nearly finished reading Joel’s books, he makes some excellent points about Windows and the rigiour of the Microsoft testing process (I’m taking it with a pinch of salt, but I’m willing to believe it, especially the amazing story about Simcity and Windows 95), so I might slow down a bit. Linux is excellent at what it does, and Windows is fine for desktop computing, so maybe I should back off a little.
However, maybe this is starting to change. I use an Apple for most thing, as do my family, but articles like this little ditty from Tim Bray entitled 2008 prediction 2: Things look bad for Windows make for interesting reading.
Wouldn’t it be cool if my separates stereo could fire data up to last.fm? CD text and a little bit of hardware, it should be possible shouldn’t it?
vlookup - really helpful. Will flesh this out later but for the moment.
Has three key fields, the value we are looking up, a table with the index key and the reference value, and a method of search.
VLOOKUP(A2,D:F,3,FALSE)
A2 is the value we are using as the index key
D:F is the table with all the index keys and reference values.
3 - is the left to right offset of the index key and the reference value
False - means “only give an exact match”
BBC iPlayer is available on Linux! Sing and dance! It’s making programs available in Flash, but it should work! Yay!
Edit: Completely forgot to add the link! It is here: Guantanamo Bay Manual diff’d
Two copies of the Guantanamo Bay working manuals appear to have leaked out. No idea how accurate they are, but by diffing they you can see all the changes. Interesting interesting: “MP”s changed for “guards”, “golfcarts” changed for “Gators”, many other things.
The boys over at 37signals are very clean thinkers, and it is because of this that they are so successfull I believe.
The idea of releasing their software as installables (to your companies server) is very appealing for many, and there is a lot of money to be made. It has lots of advantages too (your bandwidth bill for example) and I suspect that many big companies would have jumped on the idea (you can imagine the coversations “Yeah yeah yeah! Great! Sell copies for peoples offices AND run it online for the SME’s and home users! Yeah! Can you make it run on mobiles?”), and many would have developed it as installable software in the first place, but 37 signals have detailed the reasons they are not doing this, and the reasoning is a great example of some wonderfully clear thinking.
It heartens me to see “We’d be a different company” at the top of this list, this is important. The more technical reasons (data migration, unpredictable environments, the need for a helpdesk, etc etc) and all excellent.
FOLLOW UP: Joel On Software has an interesting article about why they offer FogBugz as an installable.
I always come back to the power of Unix tools, and the ideology of using many small tools in stages to achieve a goal rather than crafting a monolithic solution. Tonight was a great example how how a little automation can save a huge amount of time.
My dad has a massive stack of Excel spreadsheets called Book1.xls, Book2.xls etc etc. Each is representative of a single day, and needs renaming to reflect this. Like any good Windows user, he was opening up each one, finding the line with the date in, closing the spreadsheet, renaming Book43.xls to 07 06 2007.xls, and moving on. Ironically he was working on his Mac for this as it’s for user friendly. So - prime from some automation.
Firstly, all of the books he has renamed already needed renaming in the format YYYY_MM_DD.xls so that they can be ordered easily. Step in a bit of ksh programming.
Firstly - copy them all off to one side, and work on the copy. It is very easy to cock up and cp -pr making knocking a copy off to one side easy.
I like working from a driver file, so I did an “ls *.xls > list.txt”, then start a shell script that loops through the driver list using “for i in `cat list.txt`” and split out the three bits of the file name by piping it through awk, and spitting it out of the other side in a new order. To do this I used “echo $i | awk -F” ” ‘{print $3 “_” $2 “_” $1}’. With a little more pipe magic, I used this output to mv all the files around and job done.
Next, and more interesting, renaming all those Book*.xls to YYYY_MM_DD. The Unix command strings pulls ASCII out of binary files, so doing “strings Book1.xls” gives back a whole pages of text. One of the lines has “rubbish rubbish Date: 9 September 2007″. This is excellent news, because we can use awk again, splitting by “:”, and then select out column 2 with “awk -F”:” ‘{print $2}’. This splits back “9 September 2007″ which we can then use with awk again, splitting by spaces this time with awk -F” “, and use the same trick as the first set of files to rename the Book1.xls to $3_$2_$1 (2007_september_9.xls in this case).
OK, so far too much detail in there for the cusual user, and I imagine there are a million and one neater ways to do these things if you ask a really good hacker, but the point I was going to make was this this is only possible because of the Unix philosophy of being able to plug together little tools in a million and one combinations. All I used here was ls (to list the files), mv (to move them), awk (to separate things out separating by different characters) and cat (to read out files). The whole job took about an hour (including all the time to work out how), and saved at least 5 hours. It would scale too, saving 500 hours for the same investment of 1, and it’s just not possible on todays dumbed down Windows world. Shame.
You can’t help reading about somebody who builds their own helicopter from scrap, and be impressed. Most people in this country can’t explain how a car works, and wouldn’t even have a clue where to start.
I did have a post about how I didn’t hate Windows media Player quite as much as I did last week having used it for a little while and found some useful things, however….
As I was looking through some Unknown Artist files, I noticed some funny items in the Comments section of the mp3 tags. They look like hexadecimal numbers, here is one as an example: 00001A39 00001801 00007A8B 00008713 00030D57 00030D57 00008000 00008000 00009C85 0001ADC7
At first I wondered if these were something added for tracking mp3s through p2p networks, which would make sense. A company could add comments, and then share them, and then track their dissemination. I have some files which I’ve downloaded, so that was possible, but i checked a little further.
These numbers appear in the comments to each and every one of my mp3s! Include those I’ve ripped from CDs I own! Now I’m pretty sure cDex hasn’t added them (I’ll rip something this afternoon and see), and I have a few mp3s on my laptop that WMP has seen nothing of, so I imported one and…. the magic comment numbers appeared!
I suspect they might be a way for the internal WMP library tracks files, but surely they could be used for far more things, many of which are EVIL EVIL EVIL!
Every so often I will try to send an email from gmail and I get a pop up that says something like “It appears that you wanted to attach a mail to this mail” and gives me the OK/Cancel options.
I put this down to me accidentally clicking on attach but not specifying a file, but then this morning my paranoid mind got the better of me - what if somebody had really cunningly written something that ‘jacked Firefox and some how attached itself to you web based emails as an attachment? Could such a thing be possible - maybe, just maybe. , especially with tools such as GreaseMonkey around.
So I went of searching and it turns out to befar less exciting, it’s just Better Mail, a gmail plugin using GreaseMonkey which introduces some really nice functionality, one but of which is called “Attachment reminder”. It scans your email for words like “attach” and if they are there, reminds you to attach the file. Handy, but a little irritating if you don’t know about it!
Between my Outlook install for work, and my gmail account which aggregates the mail from about 4 or 5 pop3 addresses, I was drowning in a sea of email. I got about a thousand messages a day, of which a good 800 were spam, and of the remaining 200 I probably only had to deal with about 20 of them right now, instantly, when they arrived. Ignoring my gmail situation (which I dealt with separately), I have finally reached a state of inbox zen, and I’m working far more efficiently for it. There are some useful tips in several articles, but here is my journey.
When you’re working in a Microsoft environment, each application tends to demand to have your attention every time something happens. Each time an email arrives a little box slides into view in the bottom right hand corner of the screen, with a helpful first line of the email and the senders name. You can’t help clicking on it, it’s just instinct. I’ve tried to ignore them and if I’m really really busy it works, but most of the time I’ll stop what I’m doing and flick back to Outlook and see. AIM is the same - each time a contact logged in, up comes the status box and a little tinny sound is played to make sure I know something has happened. AIM aside (because I turned off all of the attention seeking little blighters), these constant interruptions reduce your productivity both explicitly (by dragging you away to another application) and implicitly (by breaking your concentration). After seeing the improvements in moving to word processor (one that is not a DTP package in all but name), I was determined to do something about the curse of email.
First think is first - sort out my 1500+ message inbox. In Outlook I created a set of folders. One for the current project. One for time and expense reports. One for training, holidays and other such company admin. One for Promo email from external companies. One for personal banter. I then groups all of my inbox by subject header (going on the idea that threads of emails are generally on one subject) and cleared down my inbox. There’s no magic bullet, you’ve just got to put your head down and plow through the backlog. It took a couple of hours whilst some scripts were running on one of the servers.
Thats the history sorted out, now to deal with the present. I went through my newly filled folders and created rules for the big bunches of mails along the lines of “If it’s from them, put it here”, “if it’s got this in the subject, put it here”. They don’t need to be fine detail, you can refine it later. I did run up against the problem whereby Exchange server limits the number of rules you can have (or our exchange servers do - possibly this is a customised rule). Now most of the mails I receive get filed automatically.
Now, instead of watching my inbox, I have Outlook defaulting to the Unread Items smart-list (I’m sure there is a M$ term for it, but the Apple smart-list is the only one that comes to mind), and have added the col um “folder” to the display, removing the size column (you’ll never sort this by size) and ordering by date received. As email arrive they are filed, and I can see they are new. Occasionally Outlook defaults back to the inbox which is a pain because you miss the new emails that has been squirreled away, but it happens less and less. Maybe it was just me in old habits clicking on inbox.
At the end of the working day I take 10 minutes to go through all the messages in my inbox that didn’t get auto-filtered, and file them into the correct category. I’d probably create a few more rules if I had the option.
The last three of things I did have made the biggest difference. First I switched off the automatic highlighting, and thus the little pop-up boxes in the bottom right of the screen have fucked off forever. Now I check back to my email when I’m finished doing something, gone are the interruptions.
I switched my email editor away from being Word (bloatware if ever there was one) and then switched by mail format from HTML to RTF. Ideally I’d go for plain-text, but as most people don’t switch the default font from system, my emails arrive virtually unreadable for more of the company. By not having Word fire up each and ever time I wrote an email, I don’t have the pauses, hangups and memory overhead that comes with Word. I also don’t get the highlighting of incorrect spelling, which I liked, so i turned on “force spell-check before sending”. It’s a small price.
Lastly I increased the amount of time between automatic updates. When Outlook hangs, even briefly, all of the other M$ Office applications hang also, and even the briefest flicker of hesitation causes me to alt-tab to see if my machine has hung. The majority of these brief hangups seemed to me to be when n outlook scurried off and checked the exchange server over our slightly dodgy network, and either had to wait to poll a response or suddenly had to drag a large attachment across. By increasing the time between send/receive sweeps to a full 5 minutes (from 30 seconds) these glitches have all but vanished. I don’t know if Outlook is doing some stuff in the background (I doubt it) or if the reduced frequency simply means reducing the frequency of glitches in proportion, but I barely notice them.
So now I have achieve inbox zen. My inbox is always zero at the end of the day. Email no longer gets under my skin and breaks my concentration. My other applications seem to run a little happier, and all it took was breaking “doing it the way it is always done”. Not everybody I work with likes the new regime, and I’ve had several phone calls along the lines of “but I emailed it to you 5 minutes ago!” and if it really is important, I can hit send and receive and in it pops. Normally it’s already there, and I have just been too busy to see it, and if it’s really really important, they’ll call me to demand an answer.
