Exercising on the road

I've spent the past week bouncing up and down the west coast between San Jose, CA and Portland, OR -- not spending even 48 hours in either location at any point.

This threw a wrench into my exercise program because not only did I have to find the time to exercise, I also had to figure out what to do with my sweaty clothes when I checked out each day.

At first glance, you might think that's easy -- just put the wet clothes in one of the plastic laundry bags and pack it. That is what I typically do when I'm checking out on my way home. However, since I wanted to use the clothes to exercise each day and I didn't feel like putting on wet clothes to go work out, I needed to dry them out.

When I'm staying at the same place, I can just let them air dry and that works well enough. However, since I had to change hotels 3 times this week, I needed something else to do. I could have used the iron to heat up and steam them out, but it just felt like something was wrong with ironing sweat into my clothes.

I ended up using the room blow dryer to just blow them dry. Worked fine. Clothes were dry each day and nothing appeared to be growing on them (plus the rest of my clothes stayed dry.

In case you're wondering, I did an hour on the stationary bike each day. Not too shabby for an old man, if I must say so myself.

Tags : travel / exercise

Digitizing slides

In the days of film cameras, one of the ways to take a lot of pictures cheaply was to use slide film rather than standard film. The film was around the same price, but when developing slides, you didn't get any prints, so rather than a $10 or $15 bill for the developing, the bill was just $3 or so (if I remember correctly -- in any case it was way cheaper). Others also would claim that the slide film was better for pictures & sharing since you could no project them to an audience (back in the days of 9 and 11 *inch* black and white TVs and *no* computers, there wasn't any other way to do it).

So, I have thousands of slides that I have taken over the years(several hundred from my honeymoon alone) and my mother-in-law brought over a bunch of slides that Angie's father had taken over the years (going back to the late 50s). I want to get all of these scanned into the computer so that we can share them and, if desired, print them.

Before I get into the nitty gritty, I want to lay out some ground rules that I have for scanning large batches of slides/negatives. These have grown out of my experience scanning film and your mileage may vary, but I think they are a good starting point for anybody thinking about a similar project. They include:

• I want the process as automated as possible so that I can do real work while the scanning is going on. Processes that require manual intervention every few minutes means that I have to dedicate larges amounts of spare time that I just don't have (like any of you do).
• I want "good enough" quality pictures to come out of the scan process so that I don't have to do any manual processing of the photos (other than rotating them). When I first started scanning negatives, I would do a raw scan at high resolution and then spend 15 to 20 minutes per photo to get them to a state where I liked them. This is clearly unacceptable for large amounts of photos.

  So my model is to get them good enough off the scanner so that I can enjoy/share/watch/etc. without any manual processing.

• I want to be able to easily figure out which slide/negative the photo came from after I'm done scanning in case there's a picture that I want to do more with (such as scanning at high resolution and lots of manual processing so we can print out an 8x10 or 16x20 photo). This means that I need to be able to figure out which negative from without having to resort to a manual search of thousands of slides.
• I want to preserve the film in case someone wants to work with it years from now.
• Speed is not the driving factor. Scanning thousands of slides/negatives will take time. What is key is that the work can be done while I'm doing other stuff. This leads to some choices on the scanning which actually make the scans take longer, but you get better quality scans and you get to keep working on the day job while you're doing the scanning.

These ground rules led to a number of choices I made in setting up this process. As I describe the process, I'll try to explain why and how I made these choices.

Choosing the scanner

The first issue to address is how am I going to scan slides themselves. There are two basic options for scanning slides:

• Using the slide adaptor that comes with most flatbed photo scanners (if you have a multi-function device (otherwise known as an all-in-one), you're probably out of luck as they don't seem to come with options for scanning slides). These adapters typically require that you place some number of slides (typically 3 or 4) into the adapter, remove the typical white background for document scanning and then scan the slides).

  I find this process painful for many reasons, the biggest one being that it's very time consuming and manual in nature. However, this isn't too bad if you don't have a bazillion slides to process.

• Using a film scanner designed to scan slides and negatives (film) rather than scanning documents/photos. These typically do a much better job on film that the flatbed scanners and they usually also have substantial automation capabilities.

It just so happens that I have both types of scanners and for me the clear choice was to use the film scanner. My film scanner is a Nikon Super Coolscan 4000 ED (it's about 5 years old and has been superseded by the newer 5000ED).

Organizing for scanning

If you're like most people, your slides have not stayed in their little boxes that you get back from the developer and frequently they are intermingled (in some cases within one of those slide projector trays, in other cases in the little slide shoe box where you threw all the slides).

One note about handling slides: Most slides are raw film stored within a cardboard or plastic mount which just holds the film without providing any protection to the film itself. You should use care when handling the slides to keep fingerprints, water, dust, etc. off the slides. I recommend using low-cost lint free gloves available at most photo shops when handling the slides.

You can choose to stay with the disorganization and just scan things, or you can put the slides back into their original sets. I chose to do the latter because figuring out what's on slides and telling stories about them frequently his helped by the nearby slides on the same strip of film. Getting the slides back into the set and then perusing them in order helps greatly.

To get them back into sets, you need to look at each slide. Most slides, even those printed many years ago, will have two pieces of information on each slide. A slide number in one of the corners and a processing month/year stamp. Sometimes this information is printed on the slide. Sometimes it's embossed in the cardboard mount. In many cases, the printing is hard to read and you have to use some sleuthing to figure out what set the slide belongs to and what slide number it is in that set. In the slide below you can fairly clearly see the slide number (34), but the processing date (May 89) is embossed on the cardboard and a bit harder to see.

Once I had them all grouped in sets & ordered by slide number I simply rubber banded them and put them into my to-be-done box and then started cranking.

Scanning the slides

Setting up the scanner

My 4000ED has an optional slide feeder (SF-200) which can feed up to 50 slides at a time for automated processing. This is ideal for my project. However, in many of the reviews of the product and in various support web sites, I found that there were many complaints about slides jamming in the machine -- which would really interfere with my automatic process requirement. I came close to just blindly upgrading to the latest version of the feeder (SF-210) thinking that it had to be better than the one I already had. However, from the reviews that didn't seem to be the case.

I should note that after looking at the wide variety of slides that I had in my collection (especially when I added in the older slides from my mother-in-law) it isn't so surprising that this is an issue. The slides vary greatly in materials (plastic, cardboard, even some metal) and they varied greatly in thickness.

All that said, I found one suggestion in an Amazon review that recommended tilting the scanner about 10 degrees and instead of using the spring-loaded slide pusher, place a C battery into the tray (it would roll down with the slides adding just a small amount of continuous, even, pressure). I gave that solution a whirl and across about 2K slides only had 6 or so jams -- two of which were caused by material defects in the slide mounting (the film had curved out of the mount and caught on the next slide causing the two to load simultaneously). Not bad.

To accomplish this I used two index card packs to raise the one side of the scanner and just placed the battery into the tray as you can see below:

Setting up the scanner software

Nikon Scan 4 is the software package that comes with the scanner. I modified the default settings to enable the following features:

• Enabled Digital ICE - which does a great job getting rid of dust and small scratches -- it's not perfect, but it does work pretty well.
• Enabled Digital ROC and Digital GEM post processing - these do a level of fade & color correction that makes many scans presentable that otherwise wouldn't be without a lot of manual processing.
• Enable multi-scanning 2x - each slide is scanned twice and the scanned data is averaged together -- this gets a better scanned picture on most slides.
• Set resolution to 2,000 pixels/inch (about 1/2 the full res quality of the scanner) at 100% scale. Just to keep the pictures down to a reasonable size on disk and to make some of the post processing more efficient. I can always come back later if I want a better quality scan on a particular slide.
• For each batch scan, I set the file name to a one up sequence starting with the year (so, for example, the slides I recently scanned had a base file name of si2009001 and a two digit sequential number of the slide within the slide set). When I processed the next batch, I would increase the base file name by one (e.g. si2009002). The net result is that I could tell which slide set and which slide within a slide set a digital file came from . For example, a digital file with the name si200904523.jpg came from slide 23 in the 45th slide set scanned in 2009.

Loading the slides

Emulsion side - Each slide has an emulsion side and a smooth side. The emulsion slide is the side that the image is recorded and it recorded backwards (to view the slide correctly you view through the slide from the non-emulsion side. This is important because most scanners will tell you that they want the emulsion side facing a particular way (either by directly mentioning the emulsion side, or by using pictures of a slide with an ABC on it (when ABC is backwards you are looking at the emulsion side). On most slides that have some kind of printing, the side that indicates "this side toward screen" or something like that is the emulsion side and the slide number and date stamp are typically on the viewing (non-emulsion) side.

Up vs down - the orientation of the slides (which edge is up) seems to be somewhat random with respect to the printing on the slides. In some cases they are both in sync (the slide correctly oriented when the number/time stamp are on the top. In other cases it's the opposite (the number/date stamp needs to be upside down on the bottom in order for the slide to be oriented correctly). I found I had to look at a few slides to figure out which way it worked with that set.

Landscape vs Portrait - while slides usually appear square, the film within the slide is not. When you're holding the camera horizontally (the normal position) the image will be recorded in a landscape mode (where the width of the image is longer than the height of the image). When you're holding the camera vertically (on its side) the image will be recorded in portrait mode (longer height, shorter width). This is important in slides because in most scanners you should not turn the slide to correctly orient the picture if it was taken in portrait mode. Just scan the picture in landscape mode and later, in software, rotate it 90 degrees to get it into portrait mode. The reason for this is that most scanners only scan the landscape portion of the slide and will miss some of the slide while recording some of the mount if you scan the slide in portrait mode.

Slide Numbers - most slide sets do not start with slide 1 (at least most of mine did not) and frequently that have slides missing (sometimes simply because the slide image was blank). I wanted the actual slide numbers to match the file names so I would start the file numbers with the first slide number and I would ensure that all slides were sequentially in order, filling in missing slides with slides from the end. When I had to do filling in, I would go back to the files after the set was scanned and manually renumber the fill-in slides to correctly represent their slide number.

Scanning the slides

I would simply load a set into the feeder (correctly oriented, emulsion side to the right when looking at the scanner) indicate in the software that I was feeding X slides and set the starting number at Y. Then I was off to do the real work while the scanner went along chugging through the slides in the feeder.

Slide Storage

In order to be able to quickly locate slides, as well as to provide for archival storage of the slides, I chose to use Print File Archival Slide Preserver sheets for the slides and placed a label on each sheet indicating the slide set (which was part of the digital file name) that the sheet contained:

You can get these at many photography supply stores. I purchased my at Archival USA.

Once I had the slides stored in the sheets, I placed the slide preserver sheets into Century Box Archival Storage Albums (that I also purchased from Archival USA). Another option would have been to buy the file hangers that Print File makes and simply hang the sheets in a file cabinet, but I preferred the storage box. Anyway, I placed the slide pages into the boxes and placed labels onto the boxes indicating which slide set ranges were in the box.

Miscellaneous Tidbits

Use the magnifying glass, Luke

I found having a magnifying glass quite useful in trying to determine the slide numbers and/or date stamps on slides as well as to try to determine the orientation of the slides on slides that had no markings. It was just plain useful. Get one and have it nearby when you're working on the slides.

Remounting Slides

In some cases, it might be worthwhile to remount slides. For example if the mount is damaged, too thick, or otherwise interferes with being able to scan the image. I had this with one particular set of slides that came from my mother-in-law. It seems that in the late 1950s in Europe, slides were mounted in metal mounts that sandwiched the film between two pieces of glass. When they got to me, they were in pretty sad shape:

So I ordered some slide mounts and peeled back the metal cover, separated out the film from the glass sandwich and mounted them into new slides which scanned much better than the originals had.

Summary

This process seems long and arduous, but in reality the most time consuming part (other than the remounting of that one metal set) was the organizing the slides step because many of the slides were mixed together, some had no writing on them whatsoever, many had slide numbers and date stamps that were almost unreadable (magnifying glass helped there sometimes).

Once the scanning got started, the process essentially amounted to about 5 to 7 minutes to swap slides and store the scanned slides every hour an a half or so (that's about how long it took to go through the average 30 or so slides per set with the settings I had used on the scanner software).

I'm very happy with most of the pictures and for those that I'm not happy with, the slide itself usually left a lot to be desired -- almost always because of low exposure on the film.

Tags : scanning / photograph / slide / 35MM / digital conversion / digitizing

Unsubscribing hell...

For some unfathomable reason I decided today to try to unsubscribe to some of the various spam messages I get from reputable companies. I would never try to unsubscribe to the umpteen million messages I get about body parts enlargement (some of which wouldn't look so hot on my if they were enlarged) or performance enhancement as the act of unsubscribing just confirms that they have a real person on the other end of the email line.

So, for reputable companies in the US, they are required by the CAN-SPAM act of 2003 to have an opt out method in each email. From the FTC's web site:

It requires that your email give recipients an opt-out method. You must provide a return email address or another Internet-based response mechanism that allows a recipient to ask you not to send future email messages to that email address, and you must honor the requests. You may create a "menu" of choices to allow a recipient to opt out of certain types of messages, but you must include the option to end any commercial messages from the sender.

Any opt-out mechanism you offer must be able to process opt-out requests for at least 30 days after you send your commercial email. When you receive an opt-out request, the law gives you 10 business days to stop sending email to the requestor's email address. You cannot help another entity send email to that address, or have another entity send email on your behalf to that address. Finally, it's illegal for you to sell or transfer the email addresses of people who choose not to receive your email, even in the form of a mailing list, unless you transfer the addresses so another entity can comply with the law.

So, I took a look at several of my emails... The emails from Lands End, Sears, 1-800-Flowers.com, American Express and Apple all had links and they all worked as one would expect. The either directly unsubscribed you or brought you to a page that gave you a few options (different kinds of emails, change email address, etc.) and one or two clicks and you were done.

Microsoft, on the other hand, was a true royal pain in the *ss. I received an email from them that included the unsubscribe link at the top:

And another at the bottom:

So one would think that it's all kosher. That clicking on the link would get you unsubscribed. However, that wasn't to be the case. What you got when you used that link was a page which said that I had to use my Windows Live ID to manage my settings and that if I didn't have one, I would have to create a Windows Live ID account in order to manage my subscriptions.

So you can't just unsubscribe. You have to create an account on some Microsoft server.

Being the persistent one, I went ahead and did so. That required that I provide an email address and also required out-of-band email validation (where they send you an email that has a link you have to click on to prove that you actually have that email address.

Did that and got logged into Windows Live. However, all the stuff about managing my subscription was gone and there were no clear links on the page that would get me there. So I went back to the email that started this and selected the unsubscribe link again.

This brought me to the "Profile Center" where there was a link for manage subscriptions. I thought I was getting close, but no, there was another roadblock that they threw up. There was no email address in there (they didn't take the one I entered for my Windows Live ID account). So I had to enter it again. And, of course, before I could manage it I had to go through the email validation again.

Then back to the profile page and back to managing subscriptions where I could finally unsubscribe. Now I'm stuck with a Windows Live ID account that I don't want but I don't see any easy way to get rid of it.

I think this rigmarole they have set up is in clear violation of the spirit and intent of the CAN SPAM laws and should be fixed. I should be able to unsubscribe easily without having to create an account. I should be able to unsubscribe with a minimal of effort.

Kudos to Apple, Sears, and all the rest who, IMHO, got it right. Daggers to Microsoft who clearly got it totally and inexcusably wrong.

Tags : spam / Microsoft / can-spam / Live ID / identity / email

Digitizing life

Like many people today, I have a large collection of analog media containing family memories. Much of it is my own, but a substantial portion belongs to either my or my wife's parents. This includes film negatives, slides, prints, video film, video tapes, etc.

The saddest part about this old stuff is that it deteriorates over time (even when aggressive archival storage methods are used). In addition, it's very hard to share and usually gets dispersed as various interested parties (i.e. siblings) request to take one of them (sometimes promising to make a copy and return the original -- and I'm sure some actually do that).

I have piles and piles of pretty much all of that other than video film. I have decided that it's about time to bring it all into the modern digital world and am digitizing all of it -- negatives from all the 35MM photos I took, prints from all of our kids class/sports photos or from those 4x6s that we don't have negatives for, thousands of slides (which, IMHO, were the old fashioned "digital" camera in that you just paid $3 to get the roll of film developed without any prints and then said you would print the photos you liked, but never got around to it :-)).

When I'm done, I expect to be able to share my entire digital collection with my family either directly or when I post the more interesting photos on Facebook :-). I also expect that when my kids grow up and leave the house, they will each be able to take a copy of our entire collection with them to be able to peruse whenever they like.

I'm going to write a series of blog entries describing what I've chosen to do for each type of media and how I proceeded. Hopefully some out there will find it useful in one way or another.

BTW - there are a number of services out there that will do this for you for a fee. I've chosen to do it all myself rather than use a service because I want to organize things as I convert and I want to have sensible conversions (if you used the video camera to record your kids birthday and your friends kids' school performance you don't want them on the same DVD -- at least I don't). I've also worked to automate the process as much as possible so I can do it while I'm doing other things.

Finally, I've accepted that this will take a long time and not be done overnight and I will methodically work through the piles (and they are large piles).

Wish me luck!

Tags : digitizing / scanning / film / 35mm / slides

Another change in United's Mileage Plus Program

I've written about some of the changes United made to the Mileage Plus program for 2009 (most of which I don't like), but I just noticed one that hasn't been documented much of anywhere that I have found. Kind of just snuck in there.

In the past, when you qualified for one of the premier levels, that status was good through the end of February the following year (so, my 2008 1K status was good through the end of Feb 2009). However, for my 2009 1K status my card is only good through the end of January 2010 -- a month shorter.

This is probably because with all the electronic record keeping, they think they don't need the extra month to get all the records in order to determine status.

This was not mentioned in the 2009 program changes page.

We'll see if my Global Services card (which I haven't received yet, half way through the month) has the same timeframe or if it gets the extra month when my card, hopefully, shows up.

Tags : United / ual / Mileage Plus

United comes clean on Global Services status

For years, United has been very secretive about how one becomes a Global Services member. My blog entry on Global Services is still the most popular page here, 2 years after it has been written -- accounting for more than 25% of my page hits. It's even the number one search result for "United Global Services" on Google (yeah, I'm proud :-) ).

When I logged into my united account today, I found the following published on the web site:

Ensure your Global Services status for 2009 Fly 50,000 miles on United® or United Express® in First (F, A, P), Business (C, D, Z), or full-fare United Economy® (Y or B) during 2008, and your Global ServicesSM membership will be renewed for the 2009 program year.

Track your progress by visiting united.com/gstracking.

Of course, it's kind of late for the 2009 program year at this point. But still it's now out in the open as to what you need to do to qualify. They even have a web page that you can go to to check your earnings status. Mine is still showing my 2008 earnings (since I know I have absolutely zero earnings in 2009).

Of course, I'm not convinced that this is the only way to get Global Services status. I think that their marketing and business relationship department will use GS stats as a reward for important business partners who bring them substantial corporate business, even if they, themselves, don't fly a lot. That's business as I would expect it to be.

Tags : ual / United Airlines / Global Services / travel / Mileage Plus

Situational Awareness

One of the best defenses against phishing, scamming or pretty much any other type of social engineering attack is to be aware of your situation and what to expect to have happen as well as to know when it should happen. The various attacks that come along should all raise red flags at several steps in the process. In the real world, we get this through millions of years of survival training -- those who didn't sense trouble usually died out before they could reproduce.

However, in the internet world, most of the visual and/or aural queues that raise your sense of awareness and caution are missing and we need to learn a new set of such protection mechanisms.

To that end, I'm going to periodically talk through an attack and point out things that one might notice which should cause you to think twice about continuing (or at least do a much more detailed check of whats going on before you continue).

Today, I received an interesting email reportedly from "Classmates.com" (which, of course, we all know we can't trust as anyone can claim to be anyone else with current mailing technologies):

Your Classmates Events: Reunion January 16th 2009 " With pride and joy we invite you to share a special day in our lives and join us for the Class Reunion on Friday, January 16th 2009. Bring the gang from Our High School back together again! Great party - from start to finish! " Proceed to view details: http://video.classmates.completeserv.user-v5mn1ckah.newyearclassmates.com/messages.htm?/type/INVITATION=m5kibxmz390kynf Your favorite people are already here, so use ClassmatesTM to bring them together. With best regards, Carmine Hilton. Customer Service Department. Copyright 1995-2008 Classmates Online, Inc. All Rights Reserved.

At first glance this seemed somewhat legit because I am a member of Classmates.com and so could reasonably expect to get emails from them. I'm also in a graduating class that would have an interesting anniversary in 2009 so it does make sense that we would be scheduling a reunion.

However, the email address to which the email was addressed is not the one that I have associated with classmates.com account - so clearly it wasn't classmates.com sending me the email. The address that was used is one that I've had for ages and typically gets close to 99.9% spam, so my internal "what's going on here" guard sprung up.

In addition, the email didn't look like the typical Classmates.com email -- which is just stupid laziness on the part of the attacker as it's pretty easy to fake someone else's email style, so while the email looking right isn't a good sign, having it look wrong is a big red flag.

Finally, the link in the email wasn't at the classmates.com domain (to find the actual domain you have to look at the third slash (/) in the URL and then work backwords -- the first two slashes should be right after the http: at the begining of the URL, so it's the next /). In this case it was newyearclassmates.com which should be another big red flag since it clearly was made to look like the real classmates.com domain.

If you did, somehow, follow the link, it brought up the following page:

This, too, doesn't look like the Classmates.com site -- another red flag and has no real information about what's going on. One would expect to at least have some text at this point with the name of the high school and other such information.

Instead all you have is a thing that looks like a video player application but actually is just an image and if you click anywhere on the image (like the play button or, if you're thinking of a YouTube video, the center of the video image) or on the Adobe Get media player button, the site tries to download and run a native application (an EXE). That should send big "DANGER WILL ROBINSON" shivers up your spine. Any website that tries to download an exe directly to your platform has to be treated as the enemy until proven to be a friend (no innocent until proven guilty here -- good sites rarely download EXEs directly like that without at least having some interactions with the user).

In this case the executable was Adobe_Player10.exe -- which I'm sure is a Trojan Horse which would do very nasty things to your computer at some point and it wasn't coming from Adobe's own web site, but rather from the newclassmates.com site itself -- another red flag (which, I hope, you never got because you didn't get to this stage). If you did get here and you think everything's legit, you should stop, go to the adobe web site and check version numbers or at least download the application directly from Adobe -- never download/install software that you got to through an untrusted link or from an untrusted site.

UPDATE: I've gotten 7 more of these same invites. All to different email addresses that route to me. That's another really good sign that things aren't well in Kansas and you should stay away from the email.

Moral of the story: It's a jungle out there and you've gotta watch out for yourself as there's nobody else doing it for you.

Tags : hacking / phishing / Trojan Horse / Internet Self Defense

Facebook vs DNS

Sometime back, about a couple of weeks ago, my Facebook page loads all of a sudden started getting very slow (like 20 seconds or so before the data started loading, but once it did start loading it was fast). It was only happening at Facebook (Google, WheresGeorge, Blogger, pretty much any other site) was working fine, so I thought the problem had to be at Facebook rather than on my side.

However, after it kept up for a week, I started to get irritated enough to dig into it. First I turned off my web proxy and went directly to the sites from my browser. Things worked fine then, so clearly it was an issue in my proxy. I run a Fedora Linux server at home that serves as my web proxy using the Apache HTTP daemon.

This past weekend, I started digging into the problem and spent several hours debugging, testing, searching the web and while I still don't have a clear reason as to the why, I do understand the what and have put together a somewhat nasty hack around the problem. Hopefully I will dig around and find or figure out what the problem is so that I can put in a good fix.

My first look at the server didn't show anything amiss. The httpd logs showed the accesses to Facebook with no errors. That led me to consider DNS as this felt like what you get when your DNS is timing out.

My /etc/resolv.conf file was clean and correct. Using the nslookup or dig tools, I was able to look up the names without problems and quite quickly on both my own name server as well as the name servers provided by my ISP. The system logs didn't show any problems in named or anything that looked like the firewall could be getting in the way.

However, using any other tool (telnet, wget, httpd) the name look ups would go through several failures before succeeding -- causing a substantial delay in accessing the site. This only happened with Facebook related sites (www.facebook.com and apps.facebook.com to mention two of them). The same tools, accessing any other site that I tried, had no problems and no delays.

Using strace, I could see that the first pass at the name service look ups were failing and each timing out after so many seconds before trying the next. Eventually, the tools go back and try again and the second time, the response comes back almost immediately and the tool continues. For example, "wget http://www.facebook.com" returned the following:

01     0.000106 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
02     0.000068 connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.1")}, 28) = 0
03     0.000076 fcntl64(3, F_GETFL)       = 0x2 (flags O_RDWR)
04     0.000054 fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
05     0.000042 gettimeofday({1227974358, 62163}, NULL) = 0
06     0.000048 poll([{fd=3, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
07     0.000059 send(3, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
08     0.000861 poll([{fd=3, events=POLLIN}], 1, 5000) = 0
09     4.998266 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4
10     0.000065 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.36.226.50")}, 28) = 0
11     0.000071 fcntl64(4, F_GETFL)       = 0x2 (flags O_RDWR)
12     0.000046 fcntl64(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
13     0.000041 gettimeofday({1227974363, 61621}, NULL) = 0
14     0.000046 poll([{fd=4, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
15     0.000053 send(4, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
16     0.000098 poll([{fd=4, events=POLLIN}], 1, 3000) = 0
17     2.998500 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 5
18     0.000070 connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("207.228.225.50")}, 28) = 0
19     0.000073 fcntl64(5, F_GETFL)       = 0x2 (flags O_RDWR)
20     0.000045 fcntl64(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
21     0.000043 gettimeofday({1227974366, 60548}, NULL) = 0
22     0.000045 poll([{fd=5, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
23     0.000052 send(5, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
24     0.000118 poll([{fd=5, events=POLLIN}], 1, 6000) = 0
25     5.997342 gettimeofday({1227974372, 58108}, NULL) = 0
26     0.000050 poll([{fd=3, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
27     0.000054 send(3, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
28     0.000416 poll([{fd=3, events=POLLIN}], 1, 5000) = 0
29     4.997778 gettimeofday({1227974377, 56418}, NULL) = 0
30     0.000063 poll([{fd=4, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
31     0.000055 send(4, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
32     0.000106 poll([{fd=4, events=POLLIN, revents=POLLIN}], 1, 3000) = 1
33     0.001235 ioctl(4, FIONREAD, [34])  = 0
34     0.000065 recvfrom(4, "\0079\201\202\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.36.226.50")}, [16]) = 34

As you can see, the delays come waiting for a response from the nameserver and it's not until the second try on the second name server (lines 31-34 before we get a response. You might think that this has something to do with my name server on 127.0.0.1, but that wasn't originally in my /etc/resolv.conf file until I started the debugging and the problem still occurs when I remove it.

A similar trace of the dig command shows that the first name server (whether it be 127.0.0.1 or my ISPs) resolves the name almost immediately (though dig uses a different communications method (sendmsg vs send) and different networking libraries.

Traces for wget with other host names return successfully on the first lookup.

I haven't (yet) figured out what exactly is causing this. But I have figured out two workarounds (neither of which are all that nice):

• Set one of Facebook's name servers as the first name server in my resolv.conf file (so my applications use that name server to resolve all host names.

  This does work (name resolutions worked first try and in very reasonable times). However, name servers are core trusted parties in your network access and I really don't like setting things up so that I totally trust Facebook's server for all of my outgoing name service look ups. Call me paranoid, but this one just isn't right for me.

• Add www.facebook.com and apps.facebook.com host entries to my /etc/hosts file (which is checked before name service look ups.

  This definitely works, though it does remove the usefulness of DNS from my access to Facebook (like if they change their IP address I won't know). However, it is the lesser evil of the two solutions I have found so far and so this is what I've done for now.

I'll post an update if I figure out exactly what's wrong (which I'm very unhappy about not being able to figure out so far -- I like being able to understand things and spent several hours after I had workarounds trying to figure it out to no avail).

Tags : facebook / dns / name server

Paul can't be wrong all the time

I have to say that, for once, I totally agree with Paul. In responding to a post by Ben Laurie, Paul disagrees with Ben's opinions of passwords and phishing.

Ben had said (and I'm showing a bit more here than Paul did in his response):

Well, no. If your password is unphishable, then it is obviously the case that it can be the same everywhere. Or it wouldn’t be unphishable. The only reason you need a password for each site is because we’re too lame to fix the real problem. Passwords scale just fine. If it wasn’t for those pesky users (that we trained to do the wrong thing), that is.

First off the phishability and reusability of passwords are distinct and separate issues. They have pretty much nothing to do with each other.

The primary reason one should not use the same password everywhere is that once that password is discovered at one location, then it can be reused at other locations. So, if, for example, you use the same password at Amazon, eBay, PayPal and Facebook, all one needs to do is find out your password on Facebook and then they will be able to sell things in your name on eBay, buy things in your name using PayPal and ship lots of things in your name at Amazon).

As Paul mentioned, there are many attacks to finding your password -- an administrator at Facebook could look it up in the password database, you could have a weak password that the hacker could attack via brute force (and if you're using the same password everywhere, they could use multiple sites to break the password making all/most of the anti-brute force rate limiting capabilities at a given site pretty moot). Just to name a few.

All of that said, Ben did have several good points in his post. Yes, we, as an industry, have done a terrible job in the usability of passwords. The typical user has been prompted for passwords so often and in so many places that they have no feel for when it should or shouldn't happen (one of the best personal defenses against phishing).

Personally, I think the utopia for online identity comes in with strong authentication to a small number of identity providers which assert my identity through SSO and Federation out to a large number of relying parties. Ben's point about the attacks around issuance/re-issuance of such strong credentials is very valid -- they can't be based on much weaker socially engineerable factors. The credentials will end up having to be issued with strong levels of assurance.

I also look forward to being able to login once at the start of my day and maintain that state in a reasonably secure fashion for the entire day without having to re-authenticate every few minutes or deal with "your session has been terminated for your security" when I've been sitting at the computer the entire time.

Tags : identity / authentication / assurance / federation / SSO / password / phishing

Is Sir Bonar one of Paul's aliases?

I just have to say that the article on ContactPoint written by Sir Bonar and quoted by Kim just feels like it was written by our one and only Paul.

Either Paul is writing under an alias, someone is working hard to emulate his ironic style, or somebody is writing seriously and just doesn't have an f***ing clue.

Interesting, very interesting....

Tags : identity / identity theft / security

Delayed Upgrades

One of the benefits one gets for being an elite member of United's Mileage Plus program is the ability to upgrade into the next class of service on select fares (most domestic fares qualify and some international fares qualify). Theoretically, there's also a benefit to being at a higher level in the program as your upgrades should clear sooner:

Status	Clears at
General Member	24 hours before flight
Premier Associate	36 hours before flight
Premier	48 hours before flight
Premier Executive	72 hours before flight
Premier 1K	100 hours before flight
Global Services	120 hours before flight

This used to work pretty much dependably until there were very limited seats left (the last one or two seats usually were left until boarding time).

However, this fall I've noticed that United has not been clearing upgrades, even when there are a multitude of seats available. For example, I'm on a flight tomorrow (in less than 24 hours) that has 8 of 12 seats still available for purchase but my (and presumably several other's) upgrade still hasn't cleared.

This has been pretty consistent on the last 8 or 10 flights I've been on, both domestic and international. It seems that the guys in "inventory control" (the part of United that makes the seats available for upgrade) has decided to not release any seats for upgrade until 10-12 hours before the flight.

This kind of makes the cool table of when things clear pretty useless and, to some extent, a bit of misleading marketing if not an outright lie.

Here's to hoping it's just a temporary glitch in their systems and things will get back to normal soon.

Tags : United / ual / Mileage Plus / upgrade

Paying for upgrades

United Airlines has announced a host of changes for their Mileage Plus program for 2009. Many of the changes involved increased mileage for award travel (other than domestic economy travel).

However, the worst change, IMHO, is that like American Airlines, United is now going to charge $$ (in addition to mileage) for mileage based upgrades from anything other than full fare economy tickets.

To me, a long term, very loyal 1K, million mile flyer, this really sucks. This was the one real benefit (upgrades without $$) that would drive business travelers to want to fly on the same airline. Now our business trips are going to cost as much as $1,000 if we want to upgrade both directions on an international flight.

United, I suggest you reconsider this change or, a bit selfishly, make an exception for your most loyal customers (1Ks/GSs) like you do for most other fees. Otherwise, I suggest that those of you who are flying in 2009 or early 2010 make your upgrade requests prior to July 1, 2009 (the effective date for the upgrade charges).

I also suggest that if this change bothers you, you take the time to let United know so. Recently, negative feedback about moving to pay for meals on international flights cause United to change their minds and maintain their current meal program on such flights. Perhaps we can do the same with upgrade charges.

Tags : travel / ual

Data Privacy Day

Please join the US, Canada (yeah, it's not just a blue state), and 27 European countries in celebrating second annual Data Privacy Day on January 28, 2009.

Designed to raise awareness and generate discussion about data privacy practices and rights, Data Privacy Day activities in the United States have included privacy professionals, corporations, government officials, and representatives, academics, and students across the country.

One of the primary goals of Data Privacy Day is to promote privacy awareness and education among teens across the United States. Data Privacy Day also serves the important purpose of furthering international collaboration and cooperation around privacy issues.

You can get more information, presentations, event information, etc from the Data Privacy Day web site.

Join the Facebook Data Privacy Day 2009 Group to hang with other participants and follow along with the developments.

Tags : identity / privacy

Changing Planes

It happens to me a lot more than I would like. I'm booked on an Airbus A320 only to have United change it to an Airbus 319 causing my exit row seat in row 11 to be a standard economy seat (not even an economy plus seat) -- that's why I'm not too keen on booking exit row seats nowadays -- though booking exit row seats is one of the primo perks of a United Mileage Plus Premier Executive.

However, it seems to be a much worse change when you've got a seat booked in United's new Premium International Class only to have United change the plane at the last moment and replace it with a standard configuration plane. This happened to me 3 out of 4 flights this summer between Dulles and Frankfurt.

I mean would you rather have this (the old confirguration):

Or this (the new configuration):

It felt like a big bait-and-switch to me. Show me the cool fancy new seats that are a world of difference better than the standard seats (the premium seats lie flat, have 15" screens with 100s of video on demand shows/movies, have cushy cushions, etc., etc.) and then stick me in a standard configuration without telling me till I get on the plane. No notice before hand. No chance to change to a different flight. No compensation whatsoever. Not even an "I'm sorry."

I could understand this if it happens once in a while, but 3 out of 4 flights doesn't sound like once in a while. I could also understand it more if there wasn't such a big financial benefit to United in using the standard configuration plane (they get to sell a whole lot more business and first class seats in the old configuration than in the new configuration). How do I know that United isn't simply saying "well, we've oversold business by 20%, so let's use the standard configuration plan so that we can scoop all that revenue." ? There's also the fact that United started publicly announcing that they were using the new configuration planes on Asian international routes around that time, so perhaps they moved the planes from the europ

Perhaps I should take the advice I received from my friend George (who was on the last such change with me): Just go with the flow and be happy with what life brings you. That would certainly be better for my blood pressure, but I just don't think that's me. I think United should offer some form of compensation to those who chose to fly on the plane because of the premium seating that United is heavily advertising.

I guess the only thing to learn from this experience is to not depend upon the new configuration planes until United has completed its roll out of the upgrades. Originally the conversion was to be complete in 2009, but now they are predicting 2010. So far, as of Sept 2008, they have only converted 13% of their international planes (7 of 21 767s, 5 of 24 of 747s and 0 of 46 777s).

Tags : United Airlines / ual / travel / Dulles / Airbus / A320 / A319

Smart Card hackery

This is an old video (from May of '08) and probably accomplished using an older technology smart card (theoretically easier to break), but it's still quite interesting to watch how one can peel back the layers of a smart card in order to snoop the communications going on within the components.

The related story on Wired.com gives a lot of interesting details to the ongoing cold-ware between satellite TV operators and hackers attempting to get free TV.

Tags : smart card / hacking

Cardspace, Liberty, & Intel's ICP

A couple of weeks back at DIDW 2008, I reported on a proof-of-concept that we put together at Intel where we combined Cardspace with our Identity Capable Platform (ICP) to show how ICP could extend/strengthen a cardspace deployment. While we used Cardspace in this demonstration, the code should work with any Identity Selector conforming to the Identity Selector Interoperability Profile.

For those of you who don't know, ICP is a research project we have been working on at Intel exploring how identity capabilities could be added to a platform to enhance online transactions. Our contributions to the Liberty Alliance's Advanced Client Technologies are part of that work.

In this proof-of-concept, we showed how a mythical bank (ACME Bank, of course) could provision an identity agent to the platform which was then subsequently used as the identity source for Cardspace when the user initiated a session at the bank. To Cardspace, the identity agent was a full fledged STS and had a managed card that has been provisioned into Cardspace (so, essentially, this was an off-the-shelf Cardspace deployment).

The provisioning process made extensive use of the Liberty Advanced Client Technologies protocols to securely provision the identity agent to the platform.

One might ask what exactly is an identity agent. I use the term very loosely to define any identity related agent software. In this particular case, the identity agent exposes WS-Trust and ID-WSF Provisioned Module interfaces as well as containing a SAML token generator and an ID-WSF IdP Service client (to be able to get minting assertions).

If you want to take a look at the presentation it's here. However, I have to warn you I write my presentations as something that needs speaking to and not as standalone documents.

Even better, there's going to be an encore presentation as a Liberty webcast on November 18th. I'll post the details once I get them.

UPDATE: Britta found it for me: Info/Registration for Webcast . Where would we be without Britta!

Tags : identity / Intel / Liberty / Cardspace / WS-Trust / SAML / Identity Selector / DIDW

Absentee Ballots

At last week's Liberty TEG F2F in Boston, Hubert (the guy living in French alps who just recently became a US Citizen) pointed out to the rest of us that the fall Liberty Alliance Sponsor's meeting in Tokyo is taking place the week of our presidential elections here in the US.

So, those many of you who will be attending the meeting in person should head on down to your local registrar (or however you would do it within your state/county) and register for an absentee ballot.

In Virginia, they only allow absentee voting for a limited set of reasons, none of which include "I'm more comfortable voting from home" or "I don't want to have to deal with the long lines at the local precinct." I think that they should allow anybody to use an absentee ballot, regardless of reason (even if they just feel like it). I mean, that's the point, isn't it: Get the person's vote counted.

I also don't like the fact that some/many/all places that use absentee ballots, only count them when they can make a material difference in the outcome (e.g. if the election's difference in votes is less than the total number of absentee ballots). I think that sucks. I would rather they just always count them (and perhaps start with those numbers first. Just makes sense to always count a vote. Imagine if they chose to not count a state's votes if the state's population couldn't make the difference in the outcome of a race.

In any case, if you're going to the meeting, be sure to get your ballot. This is sure to be an interesting election (though I wouldn't mind an Obama landslide -- even if that meant that they didn't count my absentee ballot).

Tags : voting / Liberty Alliance / absentee / absentee ballot / Obama

What ID-TBD means to me....

For those that don't know what ID-TBD is, it's an effort underway trying to tie the umpteen different identity efforts together into an uber identity organization. TBD as in To Be Determined (as in, we don't want to argue over the name till we get agreement on the organization and organizational structure).

My main goal here is to get out of the Liberty Alliance and away from it's exotic meeting locations like Singapore, Paris, Stockholm, Tokyo, Madrid, Sydney, Rome, etc.. I have become an active member in the Liberty 50 (those of us who have put on an extra 50 pounds or more since starting to participate in the organization). I'm probably at the head of the line and perhaps hit my peak at around 60lbs (30 or so kilos for the rest of you guys outside the US).

Yes, I blame Liberty for this (not my lack of good eating habits, my desire to have hamburgers and fries for every mean -- even breakfast -- my lack of exercise, etc., etc.). It's clearly Liberty's fault. You can see it in the pictures below:

That's me in 2001, shortly before I joined Liberty. And now, after 7 years participating in Liberty:

So by exiting Liberty and joining ID-TBD, I hope/expect to be able to loose my Liberty 50 and go back to my 2001 self. Even with just the announcement of the potential organization, I've made some progress in that direction:

This is why I am sooo supportive of the new organization. It has nothing to do with messaging convergence, coordination, consolidation or any other such mom and apple pie reason for me. I just want to get out of the Liberty 50 group!

Tags : liberty / ID-TBD

Let me count the ways

Washington Dulles airport now has 4 separate security checkpoints for non-employees. These include:

1. Regular security checkpoint. This is the old tried and true security queue on the check-in level of the airport. These are intended for use by the average traveler and frequently, especially around 4PM, has long, slow moving lines.
2. Premium security checkpoint. This checkpoint is co-located with the regular security checkpoint but it has its own dedicated queue. This queue is restricted to premium travelers (those in first/business class or those traveling on a flight where they have premium status -- such as United's Mileage Plus Premier members). This queue is typically much shorter and sometimes moves faster than the regular security queue. Dulles added premium security lines a couple of years ago.
3. Registered Traveler (Clear) security checkpoint. This checkpoint is restricted to people who have paid the annual $120 fee and subjected themselves to a background check. The registered traveler checkpoint at Dulles is managed by Clear. This checkpoint is down on the arrivals level near baggage claim 8 and is shared with the Employee checkpoint. Very short lines, quick processing (other than the time the x-ray scanner got a bag stuck in it with mine in there as well).
4. Dulles Diamond security checkpoint. This is a new checkpoint that just recently opened on the arrivals level near baggage claim 7. The signs for this checkpoint say it is only for expert travelers (2 trips/month) traveling alone, with only one carry on item and all their liquids already in bags. Theoretically these frequent travelers know what they are doing and the line can move along at a good clip. I tried this checkpoint on my trip up to Boston yesterday. There was no verification that I was a frequent traveler (though if they've read my blog, they will know). I think any single traveler could walk in there. I also verified that you can go through with a carry-on bag and computer bag (the sign says only 1 carry on item so I thought they might be restricting those of us who also bring along computer bags). So it would seem that anyone traveling alone could use this queue (and it was totally empty when I came through mid-day). Perhaps they will have tighter checks when the queue backs up once people notice it is here.

Tags : travel / Dulles / United Airlines / Registered Traveler

Slamming SAML..... NOT!

Jeff responds to my note earlier suggesting that using psudonymous identifiers adds security depth:

This is a very dangerous suggest as it implies that SAML is not secure enough without pseudonymous identifiers, the use of which makes SAML deployment a lot more complicated. Pseudonymous IDs are for privacy not security. If your system requires them to be secure, you have done something wrong. Period.

I was in no way suggesting that SAML was not secure enough. However, I am of the opinion that any SSO system (including SAML) is weaker, from a security and a privacy point of view, without pseudonyms than the same system would be if it was using pseudonyms. That doesn't say or imply that it isn't secure without them, just that it would be better with them.

And I stand by my statement that had Google used good pseudonyms across relying parties, the impact of their lack of the audience restriction would have been minimal. That isn't saying that I think a system should rely on pseudonyms as their primary security model, just that the effect would have severely reduced the impact of the error.

Tags : identity / SAML