Conor's Web Log of Esoterica

Rethinking analysis of Google's AP data capture

2010-06-12T06:03:00.000-07:00

In "Rethink things in light of Google's Gstumbler report," Kim Cameron asks that we rethink our analysis of Google's wireless data capture in light of the third-party analysis of the gstumbler data capture software. In particular he seems to have a particular fondness for the phrase "wrong," "completely wrong," and "wishful thinking" when referring to my comments on the topic. In my defense, I will say that there was no "wishful thinking" going on in my mind. I was just examining the published information rather than jumping to conclusions -- something that I will always advocate. In this case, after examining the published report, it does appear that those who jumped to conclusions happened to be closer to the mark, but I still think they were wrong to jump to those conclusions until the actual facts had been published.

I read through the entire report and have to say that the information in the report is quite different than the information that had been published at the time I expressed my opinions on the events at hand. The differences include:

We had been led to believe that Google had only captured data on open wireless networks (networks that broadcast their SSIDs and/or were unencrypted). The analysis of the software shows that to be incorrect -- Google captured data on every network regardless of the state of openness. So no matter what the user did to try to protect their network, Google captured data that the underlying protocols required to be transmitted in the clear.
We had been led to believe that Google had only captured data from wireless access points (APs). Again the analysis shows that this was incorrect -- Google captured data on any device for which it was able to capture the wireless traffic for (AP or user device). So portable devices that were currently transmitting as the Street View vehicle passed would have their data captured.

One factor that is potentially in the user's favor is that the typical wireless configuration would encourage portable devices to transmit at just enough power for the AP to hear them (devices on wireless networks do not talk directly to each other). Depending upon the household configuration, it is possible (probable?) that a number of devices would not be transmitting strongly enough for them to be detected from a vehicle out in the middle of the street. However, if Google had a big honking antenna on the vehicle with lots of gain in the right frequencies, it could have detected every device within the house.

Given this new information I would have to agree that Google has clearly stepped into the arena of doing something that could be detrimental to the user's privacy.

That said, however, we need to be a little careful about the automatic assumption that the intent was to put all of this data into some global database. In fact, the way the data was captured -- the header of every data packet was captured, many of which would contain duplicate information -- makes it clear that Google intended to do some post-processing of the data. One could hope that they would use this post-processing step to restrict the data making it into any general, world-wide database. Of course, we don't know whether or not they would do this and even if they would, they still have that raw data capture which contains information that could clearly be used to the users detriment.

In addition, the fact that we know that Google did this, doesn't preclude the fact that others can be doing this (or have already done this) without publicizing that they have done so -- especially those who do intend to use this information for nefarious purposes.

We should take this incident as a wake-up call to start building privacy into the foundations of our programs and protocols.

Tags : identity / privacy / google / SSID / Gstumbler / privacy

Kim vs. Google summary...

2010-06-07T08:47:00.001-07:00

As Kim and my ongoing blog discussion seems to have gone off on various tangents (what some might call "rat holes") I thought it best to try to bring things together in a single summary (which I'm sure will probably generate more tangents.

Lets list some of the facts/opinions that have come out in the discussion:

MAC addresses typically are persistent identifiers that by the definition of the protocols used in wireless APs can't be hidden from snoopers, even if you turn on encryption.
By themselves, MAC addresses are not all that useful except to communicate with a local network entity (so you need to be nearby on the same local network to use them.
When you combine MAC addresses with other information (locality, user identity, etc.) you can be creating worrisome data aggregations that when exposed publicly could have a detrimental impact on a user's privacy.
SSIDs have some of these properties as well, though the protocol clearly gives the user control over whether or not to broadcast (publicize) their SSID. The choice of the SSID value can have a substantial impact on it's use as a privacy invading value -- a generic value such as "home" or "linksys" is much less likely to be a privacy issue than "ConorCahillsHomeAP".
Google purposely collected SSID and MAC Addresses from APs which were configured in SSID broadcast mode and inadvertently collected some network traffic data from those same APs. Google did not collect information from APs configured to not broadcast SSIDs.
Google associated the SSID and MAC information with some location information (probably the GPS vehicle location at the time the AP signal was strongest).
There is no AP protocol defined means to differentiate between open wireless hotspots and closed hotspots which broadcast their SSIDs.
I have not found out if Google used the encryption status of the APs in its decision about recording the SSID/MAC information for the AP.

Now we get to the point where there are differences of opinion.

Kim believes that since there's no way for the user to configure whether or not to expose their MAC address and because the association of the MAC address to other information could be privacy invasive, that Google should not have collected that data without express user consent to do so and that in this case Google did not have user consent.

I believe that Google's treatment of the user's decision to broadcast their SSID as an implicit consent for someone to record that SSID and the associated MAC address is a valid and reasonable interpretation. If the user doesn't want their SSID and MAC address collected, they should configure their system to not broadcast their SSID.

Yes, even with the SSID broadcast turned off, some other party can easily determine the APs MAC address and this would clearly have potential negative impacts on the user's privacy, but that's a technical protocol issue not Google's issue since they clearly interpreted SSID silence to be a user's decision to keep their information private and respected that decision.
In "What harm can come from a MAC address?" Kim seems to argue that because there's some potential way for an entity to abuse a piece of data, that any and all uses of that data should be prohibited. So, because an evil person could capture your mac address of your phone and then drive along the neighborhood to find that mac address and therefore find your home, any use of mac addresses other than their original intent is evil and should be outlawed.

I believe that it's much better to outlaw what would clearly be illegal activity rather than trying to outlaw all possible uses. So, in this particular case, the stalker should be prohibited from using *any* means to track/identify users with the intent of committing a crime (or something like that).

Blindly prohibiting all uses will block useful features. For example, giving my device a means of establishing a location of where it is to obtain some location services without revealing to me the basis for that location is a useful feature that I have made use of on my iPhone and I don't believe that I've violated anyone's privacy in using this type of information to know where I am (to do things such as get a list of movies playing at the nearest theatre via the Fandango application).
Kim doesn't seem to have responded at all to my criticism of the privacy advocates failing to use this case as a learning experience for users to help them configure their APs in a way that best protects their privacy.

In summary, I do agree that MAC addresses could be abused if associated with an end-user and used for some nefarious purpose. However, I don't believe that Google was doing either of these.

Tags : identity / privacy / google / SSID / privacy

House numbers vs SSIDs

2010-06-06T08:22:00.000-07:00

In "Are SSIDs and mac addresses like house numbers?" Kim Cameron argues against my characterization of SSIDs and mac addresses being like house numbers:

Let’s think about this. Are SSIDs and MAC addresses like house numbers?

Your house number is used - by anyone in the world who wants to find it - to get to your house. Your house was given a number for that purpose. The people who live in the houses like this. They actually run out and buy little house number things, and nail them up on the side of their houses, to advertise clearly what number they are.

So let’s see:

Are SSIDS and MAC addresses used by anyone in the world to get through to your network? No. A DNS name would be used for that. In residential neighborhoods, you employ a SSID for only one reason - to make it easier to get wireless working for members of your family and their visitors. Your intent is for the wireless access point’s MAC address to be used only by your family’s devices, and the MACs of their devices only by the other devices in the house.

Were SSIDS and MAC addressed invented to allow anyone in the world to find the devices in your house? No, nothing like that.

Do people consciously try to advertise their SSIDs and MAC addresses to the world by running to the store, buying them, and nailing them to their metaphorical porches? Nope again. Zero analogy.

So what is similar? Nothing.

That’s because house addresses are what, in Law Four of the Laws of Identity, were called “universal identifiers”, while SSIDs and MAC addresses are what were called “unidirectional identifiers” - meaning that they were intended to be constrained to use in a single context.

Keeping “unidirectional identifiers” private to their context is essential for privacy. And let me be clear: I’m not refering only to the privacy of individuals, but also that of enterprises, governments and organizations. Protecting unidirectional identifiers is essential for building a secure and trustworthy Internet.

This argument confuses house address with house number. A house number is not able to be used as a universal identifier (I presume that there are many houses out there with the number 15, even in the same town, many times even on the same street in the same zip code (where the only difference is the N.W. and S.E. on the end of the street name).

Like SSIDs and mac addresses, the house number is only usable as an identifier once you get to the neighborhood and very often only once you get to the street.

People choose to advertise SSIDs so they themselves and others will have an easy time connecting with their network once they are within range of the AP - as evidenced by Mike's comment on my previous article (and, the reason why I have chosen to configure my SSID as broadcast). Yes, many people don't know enough to make that decision and perhaps sometimes choose to do what others might consider a wrong thing, but a) that's part of my issue with the wireless AP industry and with the privacy folks not using this as a good educational example.

So while people don't need to go to the hardware store to buy the number to put up on their house, they can, and many do, choose the electronic equivalent when they setup their AP.

House numbers are very much unidirectional identifiers used within the context of a given address (street, city, state, country, postal cod) just as SSIDs and MAC addresses are.

I will admit that there are some differences with the mac address because of how basic Ethernet networking was designed. The mac address is designed to be unique (though, those in networking know that this isn't always the case and in fact most devices let you override the mac address anytime you want). So this could be claimed to be some form of a universal identifier. However, it's not at all usable outside of the local neighborhood. There is no way for me to talk to a particular mac address unless I am locally on the same network with that device.

I do believe that a more privacy enabled design of networking would have allowed for scenarios where mac addresses were more dynamic and thus reducing the universal-ness and persistence of the mac address itself. However, that's an issue for network design and I don't think that what Google did was a substantial privacy issue for the user.

Tags : identity / privacy / google / SSID / privacy / privacy

Privacy Theatre

2010-06-04T04:26:00.000-07:00

In a series of blog articles, Kim Cameron and Ben Adida discuss Google's capturing of open access point information as part of its Street View project.

Kim's assertion that Google was wrong to do so is based upon two primary factors:

Google intended to capture the SSID and mac address of the access points
SSIDs and mac addresses are persistent identifiers

And it seems that this has at least gotten Ben re-thinking his assertion that this was all about privacy theater and even him giving Kim a get-out-of-jail-free card.

While I agree that Kim's asserted facts are true, I disagree with his conclusion.

I don't believe Google did anything wrong in collecting SSIDs and mac addresses (capturing data, perhaps). The SSIDs were configured to *broadcast* (to make something known widely). However, SSIDs and mac addresses are local identifiers more like house numbers. They identify entities within the local wireless network and are generally not re-transmitted beyond that wireless network.
I don't believe that what they did had an impact on the user's privacy. As I pointed out above, it's like capturing house numbers and associating them with a location. That, in itself, has little to do with the user's privacy unless something else associates the location with the user.
I hold the wireless AP industry responsible for the fact that many users don't have their APs setup in SSID stealth and data encrypted mode. The AP industry should have designed things so that they were encrypted by default with hidden SSIDs and required the user to do something to create an open network if they wanted to.
The user has to assume some responsibility here, though I really don't expect my mother to know how to configure encryption on an AP (nor do I expect her to know enough to know it's necessary). So I'm back to the AP industry.
And, perhaps most of all, I fault the various privacy pundits and all the news outlets who did not take this as an opportunity to teach the users and the industry about how to protect their data. Not one report that I read/saw went into any detail on how the user could protect themselves (which, if they still broadcast their SSIDs and leave their network unencrypted they are open to much worse attacks than Google capturing their SID & mac address).

Perhaps my view is contrarian for one who is somewhat active on the privacy side. However, I think it is a much more pragmatic view that will ultimately bring value to the user far beyond giving Google a hard time for capturing SSIDs and mac addresses which have little privacy value (in my opinion).

Tags : identity / privacy / google / SSID / privacy / privacy

Consent for my own software to look at my data???

2010-01-23T05:00:00.000-08:00

Here in the US (and I presume elsewhere) the annual rite of passage of doing one's taxes is upon use yet again. Once you've done something crazy like getting married, having kids or buying a house, the whole process gets more and more complicated as you try to minimize the amount of taxes you owe Uncle Sam.

I've always done my own taxes and for the past 10 or 15 years, I've used Intuit's Turbo Tax software to do so. I still can't bring myself to do the taxes online -- I can't help feeling that there's just something wrong about not keeping that data in house.

Anyway, yesterday I installed TurboTax to start working on my 2009 taxes (yeah, I'm a bit early, but I like to do it piecemeal as I receive tax reports and have spare cycles here and there).

Following the installation, I was prompted with the following consent screen:

If you look carefully, essentially the screen is asking if you give consent to release data to Intuit so that they can figure out whether they should offer you extra paid tax preparation services (paid out of your return) and/or a chance for you to get some portion of your return on a debit card. So they want to use the information on your return to market additional services to you.

What isn't clear to me from the disclosure is: are you actually giving the information to Intuit (in other words, is it being transferred to an Intuit server off-system) or is the consent is about the software that you've purchased and installed on your computer looking at the data locally.

If the former, then I think that they should explicitly say that the data is being transferred to an Intuit server as that isn't clear in the disclosure.

If the latter, why the heck is that necessary. It's my software that I purchased and it's keeping the data locally on my system. Intuit never sees the data unless I specifically send it to them for one reason or another. If this is really required by law, how does that match up with the "instant refund" or "refund anticipation loans" offered by the likes of H&R Block or Jackson Hewitt? Do they have to get you to sign a similar consent form before they can "notice" that you're getting a loan and offer to give it to you instantly (at great cost to you, of course)?

Even if it is the crazy latter situation, the consent should clearly state that the data is not leaving the system, that it is only being used by the software I just installed.

In any case, I did not consent to any data release... I'll wait for my money to show up when it shows up (assuming I even get a refund, which isn't always the case).

Tags : taxes / privacy / Intuit / TurboTax

Setting up a new ubuntu server

2010-01-10T13:43:00.000-08:00

I've been running my own mail server for close to 20 years... Through years, I've gone from Interactive Unix (how many of you remember that one!) to Red Hat Linux to Fedora Linux and now I'm moving to Ubunto (in part thanks to the strong recommendations I've gotten from friends, especially Pat Patterson.

I host several services on my server and because we're at the end of a relatively slow pipe, I use a dedicated server hosted at Superb Hosting. I use a dedicated server rather than the more typical web hosting or shared hosting because it gives me better control over my services and because I host a bunch of email domains for friends (some of which I simply forward to their current ISP and some who actually get their mail from my system.

So, I needed to setup the following services on my server:

DNS services for the 12 or so domans I manage (2 of my own and the rest friends & family).
Web server for my personal site.
Email services for something like 12 domains as well.

Sounds simple, doesn't it?

Well, it wasn't that simple, but mostly becuase a) I was learning new ways that things are done on Ubuntu vs Fedora, b) the tweaks of how I wnat to do things typically involves manual configuration changes that aren't always easily discerned from reading the man pages, and c) I like to understand the why as well as the how when doing administrative stuff so I spend a lot of time reading/reviewing/searching as I make my changes.

BTW - I'm not only willing, but actually want to do this so that I can keep my hands a bit dirty (and maintain some basic understanding of the current technologies used on servers). At work, they keep my grubby little hands far away from the system adminstartion side of the houose.

Anyway, I thougt it would be useful to document what I went through as I setup the server as it may help others trying to do the same things.

One note about the commands shown below: I logged in as root to do the configuration, so you don't see "sudo (command)" for all of the various commands. Some would say this might be a more dangerous way to configure the system and I would agree for onsey twosey administrative commands. However, for a long term session where you're doing nothing other than administrative commands, sudo just gets in the way. And yes, you need to be careful when you're logged in as root.

The following sections are presented below

OS Updates
Miscellaneous Tools
Firewall
Backup
Bind 9 (DNS Server)
Web Server
Proxies

Socks 5 Proxy
Web (HTTP/HTTPS) Proxy Server

Mail Server

Mail Clients

Secure Sockets Layer (SSL)
IMAP and POP
Authenticated SMTP
Web Server Mail Client

SPAM Filtering
The Switchover

OS Updates

First step with any new system is to ensure that I have the latest and greatest software installed -- this is expecially important on an internet visible server.

This involved running the following commands:

apt-get update         # to update the apt-get configuration/data files
apt-get dist-upgrade   # to upgrade all insalled packages to latest versions

This made sure I had the latest patches for this release of the OS. However, I wanted also to make sure I had the latest OS version. For Ubuntu, they have two development lines for servers: a somewhat frequently changing/evolving line and a more stable Long Term Support (LTS) line. Both lines get security patches regularly but LTS gets them for several years longer while the fast changing line will more frequently require you to upgrade to the latest OS version for patches.

Given what I do with the server, using the LTS line is the right thing for me to do (which is the version that was installed by my provider). So I ran the follwing commands to ensure I had the latest version:

apt-get install update-manager-core
do-release-upgrade

WHich reported that there was "No new release found" which is correct as 8.04LTS is the latest LTS.

If, on the other hand, I wanted the latest OS rev (not just the latest LTS OS rev), I could have edited the file:

/etc/update-manager/release-upgrades

and changed the line "Prompt=lts" to "Prompt=normal"

Miscellaneous Tools

As I went throught the isntallation and setup, I found a number of tools were missing that I had to install to do the things I wanted to do, so I'll list them here...

System V Configuration files
I like to use the System V commands for managing the system (including the service command to start/stop init.d services).
```
apt-get install sysvconfig
```
Make
I use a lot of Makefiles for managing the build and installation of software and packages. I was a bit suprised that my server didn't include that by default, but I presume that was because it is a server and doesn't have the development system installed either.
```
apt-get install make
```

Firewall

First thing to do is get the firewall up and running. While I plan to tightly control which services are exposed on which ports, I still feel much more comfortable having an explisit list of ports which are accessible from the internet at large. I also like to setup and test services locally while the are still blocked (including only opening up access from my own systems so I can even do remote testing without worrying about others getting into the server while it is a work-in-progress.

I use an iptables based firewall that is manually configured for the system. I've been using pretty much the same setup for years though I continuously tweak it. The script is written as an init.d service script so that I can install it there and have it automatically run it at system startup.

In addition to the typicall port protections, I also keep a blacklist of IPs for which I block all access to my server. Systems get on this list when I see that they are trying to hack into my system via repeated SSH login attempts.

The core iptables rules in the script include:

#
# Create a new chain named "filter" and "OFilter"
#
iptables -N filter                # add the new chain

#
# allow established connections
#
iptables -A filter -m state --state ESTABLISHED,RELATED -j ACCEPT

#
# if there are any ports to be dropped
#
if [ -f "${FILE_DroppedPorts}" ]; then
  grep -v "^#" "${FILE_DroppedPorts}" | while  read proto port
  do
      #
      # for non-blank lines
      #
      if [ x${proto} != x ]; then
          iptables -A filter -i eth0 -p ${proto} --dport ${port} -j DROP
      fi
  done
fi

#
# if there are any blocked IPs
#
if [ -f "${FILE_BlockedIPs}" ]; then
  grep -v "^#" "${FILE_BlockedIPs}" | while  read ip
  do
      if [ x${ip} != x ]; then
          iptables -A filter -s ${ip} -j LOG
          iptables -A filter -s ${ip} -j DROP
      fi
  done
fi

#
# allow ssh to this host from anywhere
#
iptables -A filter -p tcp --dport ssh -j ACCEPT

#
# allow HTTP/HTTPS to this host
#
iptables -A filter -i eth0 -p tcp --dport http  -j ACCEPT
iptables -A filter -i eth0 -p tcp --dport https -j ACCEPT

#
# allow SMTP, SMTPS and SMTP/TLS to this host
#
iptables -A filter -i eth0 -p tcp --dport smtp  -j ACCEPT
iptables -A filter -i eth0 -p tcp --dport smtps -j ACCEPT
iptables -A filter -i eth0 -p tcp --dport 587   -j ACCEPT

#
# allow IMAPs & POP3s to this host
#
iptables -A filter -i eth0 -p tcp --dport 993 -j ACCEPT
iptables -A filter -i eth0 -p tcp --dport 995 -j ACCEPT

#
# Allow DNS lookups to this host
#
iptables -A filter -i eth0 -p tcp --dport domain -j ACCEPT
iptables -A filter -i eth0 -p udp --dport domain -j ACCEPT
iptables -A filter -i eth0 \
             -p udp --sport domain --dport 1024: -j ACCEPT

#
# allow outgoing ftp connections
#
iptables -A filter  -p tcp --sport 21 \
              -m state --state ESTABLISHED -j ACCEPT
iptables -A filter -p tcp --sport 20 \
              -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A filter -p tcp --sport 1024: --dport 1024:  \
              -m state --state ESTABLISHED -j ACCEPT

#
# let people ping us
#
iptables -A filter -p icmp -j ACCEPT

#
# Log all else
#
iptables -A filter -j LOG

#
# drop all else
#
iptables -A filter -j DROP

#
# install the input and output filters for input transactions
#
iptables -A INPUT   -j filter

If you're interested, you can download the script and associated files here.

Note that at this point, while I'm setting up the system, many of those ports opened above are commented out and then, as I install the various components (such as Apache2) I open the respective port.

Once completed, I installed the script in /etc/init.d using the install directive in my Makefile (make install) and then used the following command to setup the necessary /etc/rc*.d files to ensure the firewall started as necessary when the system was booted.

update-rc.d firewall defaults

Backup

Whether or not we actually do it, we all know that we should be backing up our systems and data. This is especially true for remote systems where we don't have easy local access.

My hosting provider does have backup options, but they cost re-occuring money that I don't want to spend if I don't have to. So, my solution is to backup my remote server onto one of my home servers (where I have TBs of space anyway).

Since I have a firewall at home, I have to do the backup across a two step ssh tunnel similar to what I decribed in Backing up using S SH & Rsync. The first connection goes from my remote server to my firewall and the second connection goes from the remote server through the first connection to my backup server. I then rsync a number of directories on the remote server to the backup sever including:

/etc, /var, /usr/local, /home

For security reasons, I require private key authentication for this connection on both my gateway and my backup server, I use a user account which has no login shell and no login directory and I configure that the only service that can be accessed is the rsync service. Not perfect, but it's good enough that I can get some sleep at night.

One problem with this setup is that the second ssh tunnel connects to a port on localhost in order to establish the connection to the remote system which can be a problem if there's other ssh connection tunnels setup similarly. To get around that, I add an alias for my backup server to the localhost entry in /etc/hosts file. So, rather than connecting to localhost the second tunnel connects to the host backup_server and thus keeps all of the SSH host keys separate.

If you're interested, you can download a modified (I removed any credentials & system names) of the script from here.

Bind9 (DNS Server)

I host DNS for a most of the domains for which I host mail (a few of my friends host their own DNS, but use my mail server). A long time ago, I wrote a shell script that creates the necessary configuration files for the set of domains I manage (which makes it easy to add new domains which are following the same rules and makes it easy to change things around when I change my configuration).

Preparation for the move

Since nameserver changes can take some time to propogate through the internet, this is the first service that I installed, configured and exposed on the new system. In preparation for the move, I went to my old nameserver and cranked down the caching settings for the domains I hosted there in order to reduce the propagation time. My typical settings are:

@     IN    SOA     mydomain.com.  postmaster.mydomain.com. (
      2010010200      ; serial number
      86400           ; refresh every 24 hours
      3600            ; retry after an hour
      604800          ; expire after 7 days
      86400           ; keep 24 hours
)

In preparation for the move, about a week in advance I reduced these settings to:

@     IN    SOA     mydomain.com.  postmaster.mydomain.com. (
      2010010800      ; serial number
      3600            ; refresh every hour
      1800            ; retry after a half hour
      7200            ; expire after 2 hours
      3600            ; keep 1 hour
)

And finally, the day before the switch, I moved to:

@     IN    SOA     mydomain.com.  postmaster.mydomain.com. (
      2010010900      ; serial number
      1800            ; refresh every half hours
      1800            ; retry after a half hour
      600             ; expire after 10 mins days
      600             ; keep 10 mins
)

Installation and configuration

I installed the nameservice daemon software and utilities using:

apt-get install bind9 dnsutils bind9-doc resolvconf

I then copied my setup files from the old server to the new server. The way that /etc/named.conf is managed has changed. On my old server all of the settings were in that one file. However, in Ubuntu, that file is intended to be unchanged and the local options are supposed to be placed into /etc/named.conf.options while the host references are intended to be placed into /etc/named.conf.local. So I changed my scripts to match the new model and modified the Makefile to correctly installe the new compoonents.

I've always run my named (the nameserice daemon) within a chrooted environment and every time I do this I have to yet again figure out what pieces need to be there in order to get things working. So this time, I wrote a CreateChroot.sh script and ran it to create the chroot environment for me (and now I don't have to figure it out from scratch the next time!). In addition to creating the chroot environment, I had to change the OPTIONS directive in /etc/default/bind to include "-t /var/cache/bind" so that the file now looks like:

OPTIONS="-u bind -t /var/cache/bind"
#OPTIONS="-u bind"
# Set RESOLVCONF=no to not run resolvconf
RESOLVCONF=yes

In first setting up the new server, I made no changes other than to add a new entry for my new server. So my new nameserver had pretty much the same host entries that were on the old server. So I ran my script for creating and installing my named configuration and restarted the bind9 service.

At this point, I opened the DNS TCP & UDP ports on my filewall so that I could accept incoming nameservice requests. In order to test the service, I went to my old server and used nslookup to test the new server:

# nslookup
> server newns.mydomain.com
Default server: newns.mydomain.com
Address: 192.168.169.11#53
> www.mydomain.com
Server:         newns.mydomain.com
Address:        192.168.169.11#53

Name:   www.mydomain.com
Address: 192.168.169.11
> mail.mydomain.com
Server:         newns.mydomain.com
Address:        192.168.169.11#53

Name:   mail.mydomain.com
Address: 192.168.169.11
>exit

This showed that things were working as I intended.

The Switchover

At this point, everything was ready to go, so I went to my domain registry (Network Solutions) and changed the host records for my nameservers to make the new nameserver my primary dns server and my old server to be the secondary server.

This worked fine (though they warned me it could take 72 hours to propagate) and I ran a bunch of tests from my home network, my work network and my old server and everything was peachy keen.

Web Server

I run a web server for my own family web site. It's all hand-coded html (yeah, kinda old fangled, but I haven't had the time, energy or inclination to re-architect it. Setting it up on the new server was pretty simple.

First step was to copy over the directory heirarchy from the old server to the new server. Just tar'd it up and scp'd it over to the new server and untar'd it within the /home/www directory.

Next step involved geting apache2 installed...

apt-get install apache2

The configuration for the web servers is located in: /etc/apache2/sites-available which comes with a single default file. I renamed this file to be www.cahillfamily.com (allowing for more sites at some point in the future) and editet that file to match up the settings from the old server.

Server Side Includes (SSI)

SSI is a capability on the server which allows an html file to include html from other files on the same web server. I use this feature extensively to maintain a consistent menu structure by placing it in one file and including it in all the html files on the server.

To enable this feature, I did the following:

Set the Includes option within the configuration section for my virtual host.
Set the +XBitHack option as well. This allows me to indicate to the server that there's an include directive in the file by simply setting the executable bit on the file (rather than having to have a particular suffix on the html file).
Enabled mod-include by running the following command:
```
a2enmod include
```

Proxies

I run a few proxy severs on my remote server that I have found useful when I'm behind some crazy firewalls or when an ISP has tight controls on the number of outgoing connections -- I've run into racheted down connection limits on my former home ISP (RoadStar Internet and at some hotels while on the road.

So I setup the proxies on my remot server, SSH to the server and then tunnel my services through that server.

WARNING: You have to be very careful when you setup proxies so that you don't end up creating an open proxy that others can use to make it appear that bad things are coming from your server. If you do set one up, do so carefully.

Socks 5 Proxy

Socks 5 is used for proxying many of my different Instant Messenger connections (I have like 5 of them). For Ubuntu, the common/best one seems to be the Dante-Server wich I installed using:

apt-get install dante-server

I configured it to only allow connections from the local system (since I will have an SSH tunnel to the server). This prevents others from using it unless they have internal access to my server.

*** /etc/danted.conf.orig       2009-12-31 11:29:41.000000000 -0500
--- /etc/danted.conf    2009-12-31 11:39:16.000000000 -0500
***************
*** 37,43 ****

# the server will log both via syslog, to stdout and to /var/log/lotsoflogs
#logoutput: syslog stdout /var/log/lotsoflogs
! logoutput: stderr

# The server will bind to the address 10.1.1.1, port 1080 and will only
# accept connections going to that address.
--- 37,43 ----

# the server will log both via syslog, to stdout and to /var/log/lotsoflogs
#logoutput: syslog stdout /var/log/lotsoflogs
! logoutput: syslog

# The server will bind to the address 10.1.1.1, port 1080 and will only
# accept connections going to that address.
***************
*** 45,54 ****
--- 45,58 ----
# Alternatively, the interface name can be used instead of the address.
#internal: eth0 port = 1080

+ internal: 127.0.0.1 port=1080
+
# all outgoing connections from the server will use the IP address
# 195.168.1.1
#external: 192.168.1.1

+ external: xx.yy.zzz.aaa
+
# list over acceptable methods, order of preference.
# A method not set here will never be selected.
#
***************
*** 57,66 ****
#

# methods for socks-rules.
! #method: username none #rfc931

# methods for client-rules.
! #clientmethod: none

#or if you want to allow rfc931 (ident) too
#method: username rfc931 none
--- 61,70 ----
#

# methods for socks-rules.
! method: username none #rfc931

# methods for client-rules.
! clientmethod: none

#or if you want to allow rfc931 (ident) too
#method: username rfc931 none
***************
*** 106,112 ****
# can be enabled using the "extension" keyword.
#
# enable the bind extension.
! #extension: bind


#
--- 110,116 ----
# can be enabled using the "extension" keyword.
#
# enable the bind extension.
! extension: bind


#
***************
*** 162,167 ****
--- 166,178 ----
#     method: rfc931 # match all idented users that also are in passwordfile
#}

+ #
+ # Allow any connections from localhost (they will get here via SSH tunnels)
+ #
+ client pass {
+       from: 127.0.0.1/32 port 1-65535 to: 0.0.0.0/0
+ }
+
# This is identical to above, but allows clients without a rfc931 (ident)
# too.  In practise this means the socksserver will try to get a rfc931
# reply first (the above rule), if that fails, it tries this rule.

Web (HTTP/HTTPS) Proxy Server

Since I already had the web server up and running, setting up a web proxy was easy. First I had to ensure that the necessary modules were installed and enabled:

a2enmod proxy
a2enmod proxy-connect
a2enmod proxy-ftp
a2enmod proxy-http

Then I edited the /etc/apache2/httpd.conf file and added the following entries:

ProxyRequests On

<Proxy *>
  AddDefaultCharset off
  Order deny,allow
  Deny from all
  Allow from 127.0.0.1
</Proxy>

AllowConnect 443 563 8481 681 8081 8443 22 8080 8181 8180 8182 7002

The AllowConnect option is necessary if your're going to proxy other connections (such as HTTPS). Most of those numbers are legacy from some point in the past. The really necessary one is 443 (for HTTPS), some of the 8xxx ones were from when I was doing some web services testing from behind a firewall at work (so I could invoke the web service endpoint from my test application). Not sure about all the others, but I'm not to worried about it since I only accept proxy requests from the local system.

Mail Server

Setting up a mail server can be somewhat complex, especialy when you throw in the fact that I was moving a running mail server to a new system and adding new client capabilities. On my old server, all of my users had to SSH into my server with private key authetnication and then tunnel POP & SMTP over the SSH connection. This could be a pain (to say the least) and restricted access for clients like the iphone or other devices. Most of my users (family & friends) are using an ssh tunnelling product from VanDyke that was discontinued back in 2004.

Installation

First step is to install the necessary components. Some of these were already installed with the server OS package (e.g. Postfix) but there's nothing wrong with making sure...

apt-get update
apt-get install postfix
apt-get install courier courier-pop-ssl courier-imap-ssl courier-doc 
apt-get install spell mail

Before I start actually accepting and processing mail, I thought it best to get the clients protocols all working, so onto the clients.

Mail Clients

I needed to enable support for your typical mail clients such as Outlook and Thunderbird (which require IMAP or POP3 to retrive mail and SMTP to send mail) as well as web browser clients. In the past, I have not supported web clients and I have required mail clients to tunnel their POP3 & SMTP over ssh tunnels. With the new server, I wanted to allow access without requiring ssh tunnels so that other clients (such as my iPhone) that didn't have ready support for ssh tunneling could get to the mail server. I also wanted to add browser based support so that people could check their email from other locations (such as a friends computer).

This involved the following steps:

Secure Sockets Layer (SSL)
IMAP and POP
Authenticated SMTP
Web Server Mail Client

Secure Sockets Layer (SSL)

For remote access to my server I needed to enable SSL so that user credentials were protected. My intent was to enable SSL on all the standard mail client protocols (SMTP, IMAP and POP) and to enable browser based access to mail via HTTPS and a web server based mail client.

Certificate Generation

In order to support SSL, I needed to get an SSL certificate. I could have created my own certificate and signed it myself, but that would have lead to error messages from the clients telling my users that perhaps they shoudln't trust my server. Instead, I signed up for an SSL certificate from GoDaddy which was running a special for $12.95/year for up to 5 years.

In order to create the certificate, I had to generate my private key and then a certificate signing request using the following commands:

*** make sure openssl is installed
# apt-get install openssl

*** Generate 4096 bit RSA server key
# openssl genrsa -des3 -out server.key 4096
Generating RSA private key, 4096 bit long modulus
.............................................................................++
................................................................................
......................................++
e is 65537 (0x10001)
Enter pass phrase for server.key: abcd
Verifying - Enter pass phrase for server.key: abcd

*** Generate certificate signing request for server key (note that the
*** "Common Name" must be the name of the host that the clients will connect
*** to if you don't want to get ssl errors)
# openssl req -new -key server.key -out server.csr
Enter pass phrase for server.key: abcd
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]: US
State or Province Name (full name) [Some-State]: Virginia
Locality Name (eg, city) []: Waterford
Organization Name (eg, company) [Internet Widgits Pty Ltd]: Cahills
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []: mail.cahillfamily.com
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

At this point, I took the server signing request, server.csr, and sent it to GoDaddy to get them to sign it and create my certificate. If, on the other hand, I wanted to do a self-signed certificate, I would have performed the following steps:

*** sign the csr using our own server key (making this a self-signed cert)
# openssl x509 -req -days 1825 -in server.csr \
  -signkey server.key -out server.crt
Signature ok
subject=/C=US/ST=Virginia/L=Waterford/O=Cahills/CN=mail.cahillfamily.com
Getting Private key
Enter pass phrase for server.key: abcd

To test this, I configured Apache2 to support SSL and tested access to https://mail.cahillfamily.com. I first needed to enable the SSL module using the following command:

a2enmod ssl

I took the server key and server certificate and place them into a secure non-standard location (no need to advertise where) and set the access modes on the directory to restrict it to root only. In order for the server key to be used without a pass phrase, I ran the following commands to remove the pass phrase from the file:

mv server.key server.key.safe
openssl rsa -in server.key.safe -out server.key

I copied the default Apache2 site file into one for mail.cahillfamily.com and set it up using the following commands:

cp /etc/apache2/sites-available/default /etc/apache2/sites-available/mail.cahillfamily.com
ln -s /etc/apache2/sites-available/mail.cahillfamily.com /etc/apache2/sites-enabled/mail.cahillfamil
y.com

I then edited the configuration file to enable SSL and to point to the newly installed certificate and key files:

NameVirtualHost *:443
<VirtualHost *:443>
      ServerAdmin webmaster
      ServerName mail.cahillfamily.com

      DocumentRoot /home/www/mail.cahillfamily.com
      ErrorLog /var/log/apache2/error.log

      # Possible values include: debug, info, notice, warn, error, crit,
      # alert, emerg.
      LogLevel warn

      CustomLog /var/log/apache2/access.log combined
      ServerSignature On

      SSLEngine On
      SSLCertificateFile /path-to-ssl-files/server.crt
      SSLCertificateKeyFile /path-to-ssl-files/server.key
</VirtualHost>

I also wanted to automatically redirect any http: access to mail.cahillfamily.com to https: access, so I added the following section to the default site file which uses the RedirectPermanent directive to automatically redirect access on port 80:

<VirtualHost *:80>
      ServerAdmin webmaster
      ServerName  mail.cahillfamily.com
      RedirectPermanent / https://mail.cahillfamily.com
</VirtualHost>

IMAP and POP

After poking about some, I came to the conclusion that the right mail server for me to use to expose IMAP and POP interfaces for my mail clients is the Courier Mail Server.

Courier requires that you use the MailDir structure for user mailboxes while Postfix uses the mbox structure by default. So I changed Postfix to use the MailDir structure by adding the following setting to /etc/postfix/main.cf:

home_mailbox = Maildir/

I manually created an empty Maildir structure for all my user accounts.

For SSL, Courier requires the key and the certificate to be in a single .pem file. So I concatenated server.key and server.crt into a single server.pem file.

I edited the /etc/courier/imapd-ssl file to make the following changes:

Set SSLPort to 993.
Set both IMAPDSSLSTART and IMAPDSTARTTLS options to YES to allow both IMAP over SSL and TLS within IMAP (the latter being a TLS session that's started from within the IMAP session while the former is a plain IMAP session over an SSL tunnel).
Set IMAP_TLS_REQUIRED to 0 so that local connections from the web mail server could make use of imap without having to do TLS on the local (same system) connection. I planned to still block the standard IMAP port (143) in the firewall, so remote clients would not be able to access their mail without SSL/TLS).
Set TLS_CERTFILE to point to the recently created server.pem file.

I edited the /etc/courier/imapd file to make the following changes:

Added "AUTH=PLAIN" to the IMAP_CAPABILITY setting so that plain text authentication is allowed on non-tls connections to the imap server. This is necessary for the local connection from some web server mail clients which don't come with support for CRAM-MD5 or other non-PLAIN authentication mechanisms.

I edited the /etc/courier/pop3d-ssl file to make the following changes:

Set SSLPort to 995.
Set both POP3DSSLSTART and POP3DSTARTTLS options to YES to allow both POP3 over SSL and TLS within POP3 (the latter being a TLS session that's started from within the POP3 session while the former is a plain POP3 session over an SSL tunnel).
Set POP3_TLS_REQUIRED to 0 so that local connections from the web mail server could make use of imap without having to do TLS on the local (same system) connection. I planned to still block the standard POP3 port (110) in the firewall, so remote clients would not be able to access their mail without SSL/TLS). However, this would enable my existing clients which ssh to the server and then use non-TLS POP to still be able to get their email.
Set TLS_CERTFILE to point to the recently created server.pem file.

Restarted the courier related services:

service courier-imap stop
service courier-imap-ssl stop
service courier-pop stop
service courier-pop-ssl stop
service courier-imap start
service courier-imap-ssl start
service courier-pop start
service courier-pop-ssl start

Yeah, I probably could have simply used the "restart" command on each of them but I wanted to have them all stopped and then start them all so I was sure that they call came up cleanly under the same configuration.

Now it was time to test things. First a quick telnet connection to the local imap port (143):

# telnet server 143
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THRE
AD=REFERENCES AUTH=PLAIN SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Cop
yright 1998-2005 Double Precision, Inc.  See COPYING for distribution informatio
n.
01 LOGIN username password
01 OK LOGIN Ok.
0000 logout
* BYE Courier-IMAP server shutting down
0000 OK LOGOUT completed
closed

So that worked. I ran a similar test for POP3 which also worked. Now I was ready for some remote testing. First step was to go back to my firewall and open ports 993 (IMAPS) and 995 (POP3S) to allow incomming connections to the IMAP and POP services.

Then I went to http://www.wormly.com/test_pop3_mail_server and ran several tests with the POP3S implementation (with test accounts, of course) which all worked fine.

I didn't see a similar testing tool for IMAP, so I ran some tests from one of my home computers using the following command:

openssl s_client -crlf -connect mail.cahillfamily.com:993

Which worked like a charm (with some finagling with the /etc/hosts file to override mail.cahillfamily.com's IP address). This also worked like a charm, so at this point I figured I had IMAP and POP up and running.

Authenticated SMTP

When setting up an SMTP server, you have to be very careful that you don't configure your server as an open relay (where it will send mail from anyone to anyone). It seems that hackers, scammers and spammers are forever looking for new open relays that they can use to send out spam and shortly after opening an SMTP port on the internet you can usually find attempts to make use of the server as a relay.

For basic unauthenticated SMTP (e.g. where there's no local user authentication within the SMTP session), I configured the server to only accept incomming mail whose delivery address is within one of my managed domains. Any mail with a destination address outside of my domain is rejected before we accept the mail message itself.

However, that configuration wouldn't work very well for my users who typically do want to send mail to people outside of my domain. In the past, my solution was simple: ssh tunnel to my host then sent mail via SMTP on the local host interface where I could treat any local connections as, by default, authenticated.

While I am continuing to allow that configuration with the new server setup, it wouldn't work for those users trying to use a mail client without the ssh tunnel. So I had to enable authenticated SMTP and I had to configure it to require such sessions over SSL.

The SMTP server is managed by Postfix itself. So first step was to modify the /etc/postfix/main.cf configuration file to only accept main with recipients in my networks:

#
# restrict smtp operations on unauthenticated (port 25) connections
#
smtpd_recipient_restrictions = permit_mynetworks,reject_unauth_destination

Then I modified the /etc/postfix/master.cf configuration file to enable both TLS within SMTP sessions and SMTP over SSL/TLS by including the following directives:

submission inet n       -       -       -       -       smtpd
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
-o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
smtps     inet  n       -       -       -       -       smtpd
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
-o smtpd_recipient_restrictions=permit_sasl_authenticated,reject

These settings, along with the base configuration, should give me server to server SMTP on port 25 and client to server user authenticated SMTP over TLS/SSL on ports 465 and 587.

Now that I have SMTP which allows for authentication, I had to install and configure the sasl authentication daemon as follows:

I installed the package using:
```
apt-get install libsasl2 sasl2-bin
```
I edited the /etc/defaults/saslauthd to make the following changes:
- Set START=yes so the daemon will start.
- Configured saslauthd to place it's runtime information underneath the postfix chroot environment by changing the OPTION parameter and adding the following lines:
```
PWDIR="/var/spool/postfix/var/run/saslauthd"
OPTIONS="-c -m ${PWDIR}"
PIDFILE="${PWDIR}/saslauthd.pid"
```

I created the saslauthd run directory using:

mkdir -p /var/spool/postfix/var/run/saslauthd

Configured saslauthd to leave its files readable by postfix (so postfix could communicate with the daemon) using the following command:
```
dpkg-statoverride --force --update --add root sasl 755 \
              /var/spool/postfix/var/run/saslauthd 
```
Created /etc/postfix/sasl/smtpd.conf file and added the following lines:
```
pwcheck_method: saslauthd
mech_list: plain login
```
Restarted both saslauthd and postfix

Now I was ready to start testing, so I went back to my firewall and opened ports 25 (SMTP), 465 (SMTP over SSL) and 587 (TLS within SMTP) so that I could start testing.

To test all of this you could use a mail client, or if you're a bit more adventurous (and want to see exactly what's going on) you can do this manually within a telnet/openssl connection). The following is an example test session:

$ telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 mail.cahillfamily.com ESMTP Postfix (Ubuntu)
ehlo mail.cahillfamily.com
250-mail.cahillfamily.com
250-PIPELINING
250-SIZE 10240000
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
mail from: user@localhost
250 2.1.0 Ok
rcpt to: someuser@someotherhost.com
250 2.1.5 Ok
data
354 End data with .
Subject: Test message

Sending yet another test... hope it gets there...

.
250 2.0.0 Ok: queued as B28C461A0A9
quit
221 2.0.0 Bye
Connection closed by foreign host.

That's a standard, unauthenticated SMTP session. I find using the manual sesssion for testing makes it easier to identify what the problem is when there is a problem. For example, in the list of responses after my "ehlo" command, you see "250=-STARTTLS" - this indicates that TLS is enabled within the server).

To test an authenticated SMTP session, you will need to enter a command similar to the following (I usually do this right after the "ehlo" command, though I'm not sure if it has to be exactly there):

auth plain AHVzZXJpZABwYXNzd29yZA==
235 2.7.0 Authentication successful

The "AHVzZXJpZABwYXNzd29yZA==" parameter is a base64 encoding of a plain text SASL authentication string. You can generate one manually using the following perl command:

perl -MMIME::Base64 -e 'print encode_base64("\000userid\000password")'

Where userid = the test user's id and password = the test user's password If you have a special character in either string (such as an @ in the user id (e.g. user@host) you need to escape the character (e.g. "\@").

So, now that I have all that, I ran the following tests:

Test local unauthenticated SMTP connection to send mail to remote system (for my clients that ssh to server and send out from there)
```
telnet localhost 25 
```
and then run through SMTP session described above.
Test remote unauthenticated SMTP connection doesn't allow sending mail to remote locations. Go to a remote system and run:
```
telnet mail.cahillfamily.com 25
```
and try SMTP session above - should fail with either a) permission denied or with relay access denied when you enter the "rcpt to" command.
Test remote unauthenticated SMTPS connection as follows:
```
openssl s_client -starttls smtp -crlf -connect mail.cahillfamily.com:587
```
and try SMTP session above - should also fail, this time with permission denied since we only setup authenticated SASL connections on this port.
Test remote authenticated SMTPS connection using the following:
```
openssl s_client -starttls smtp -crlf -connect mail.cahillfamily.com:587
```
and this time include the "AUTH PLAIN" command at the start of the session. This should succeed.
Test remote authenticated SMTP over TLS connection as follows:
```
openssl s_client -crlf -connect mail.cahillfamily.com:465
```
and include the "AUTH PLAIN" command at the start of the session. This should succeed.

Web Server Mail Client

For browser clients, there are a couple of obvious possibilities that come to mind:

SqWebMail - a component of the Courier Mail Server which provides access to mail files via direct access to the mailboxes.
Squirrel Mail - a web server based mail client that gets lots of good recommendations as being one of the best open source solutions. This tool uses the IMAP interface to access the user's mail files rather than direct manipulation.
As a bonus, this tool also has an available Outlook-like plug-in that gives users the look/feel of Outlook 2003.

I took a look at the two tools and decided to go with Squirrel Mail and, for now, just install the basep toolkit. I'll explore the Outlook model at some point in the future. Ubuntu has SquirrelMail available as a standard package so I installed it using the following command:

apt-get install squirrelmail

I then modified the /etc/apache2/sites-available/mail.cahillfamily.com configuration file to use the squirrelmail application as the document root, so my users go straight into the application when they visit mail.cahillfamily.com in a browser. The modified file looks as follows:

NameVirtualHost *:443
<VirtualHost *:443%gt;
      ServerAdmin webmaster@localhost
      ServerName mail.cahillfamily.com

      DocumentRoot /usr/share/squirrelmail
      ErrorLog /var/log/apache2/error.log

      # Possible values include: debug, info, notice, warn, error, crit,
      # alert, emerg.
      LogLevel warn

      CustomLog /var/log/apache2/access.log combined
      ServerSignature On

      SSLEngine On
      SSLCertificateFile /etc/ssl/server.crt
      SSLCertificateKeyFile /etc/ssl/server.key

Include /etc/squirrelmail/apache.conf

</VirtualHost%gt;

Used by browser to test it and everything seems kosher.

SPAM filtering

To filter or not to filter.... For many years I ran my server with no server-side filtering and instead relied on client filtering. However, the abundance of crap that keeps on coming only seems to grow exponentially every year and I finally convinced myself that not only was server side filtering necessary, but it was mandatory. This is especially evident when you're trying to download mail after having been disconnected for a day or so and find that you have hundreds of email messages, most of which are clearly SPAM.

I use spamassassin for spam filtering. Looking around at most of the how-to's/docs I see that most people recommend usiing spamassassin to just flag spam, but then go ahead and deliver it to the user's mailbox. This is probably the best solution if you don't want to lose any potential emails that have incorrectly been marked as SPAM. However, that means that my clients have to download hundreds of spam messages just to throw them out when the got to the client.

For my system, I'd rather have Spamassassin get rid of at least some spam and then let some of the questionalbe stuff through. So, I've setup things such that mail messages that get a Spamassassin grade of 10 or higher get saved off into a directory on the server (one directory for each day to ease management). For messages that have a grade between 5 and 10, the subject gets re-written to include a SPAM indicator, but the message is still delivered to the intended recipient.

I've been doing it this way for the past 2 years. We get on the order of five thousand (yeah: 5,000) messages culled this way each day and I've yet to find or get a report of any false positives. Note that there's still a bunch of email that gets through with grades between 5 and 10.

Anyway, to set this up on the new server:

Install the latest version of spamassassin using:

apt-get update
apt-get install spamassassin spamd

Installed spamchk script (not sure where I originally got it, but I've been using it on my old mail server for several years now) in /usr/local/bin/spamchk
Created /var/spool/postfix/spam/save and /var/spool/postfix/spam/tmp directories for processed messages
Edited the /etc/postfix/master.cf file to add an output filter for mail coming in the default smtp connection (we don't need it on the SSl connections since they are authenticated) and to add the spamck invocation. Modified lines look as follows:
```
smtp      inet  n       -       -       -       -       smtpd
  -o content_filter=spamchk:dummy
```
And at the end of the file, added:
```
#
# SpamAssassin check filter
#
spamchk   unix  -       n       n       -       10      pipe
  flags=Rq user=spamd argv=/usr/local/bin/spamchk -f ${sender} -- ${recipient}
```

By default, Spamassassin places a detailed spam report in any message that is flagged as spam (spam score >= 5) and moves the original message to an attachement. I find this cumbersome and so instead I like to flag the subject of the message with a "[SPAM]" flag and otherwise leave the message alone (you do still get the Spamassassin headers added to the message, but they are hidden from the default view in most mailers).

To achieve this, I edited the /etc/mail/spamassassin/local.cf file and make the following changes:

*** local.cf.orig       2010-01-02 10:45:58.000000000 -0500
--- local.cf    2010-01-09 20:54:46.000000000 -0500
***************
*** 9,21 ****

#   Add *****SPAM***** to the Subject header of spam e-mails
#
! # rewrite_header Subject *****SPAM*****


#   Save spam messages as a message/rfc822 MIME attachment instead of
#   modifying the original message (0: off, 2: use text/plain instead)
#
! # report_safe 1


#   Set which networks or hosts are considered 'trusted' by your mail
--- 9,21 ----

#   Add *****SPAM***** to the Subject header of spam e-mails
#
! rewrite_header Subject [SPAM]


#   Save spam messages as a message/rfc822 MIME attachment instead of
#   modifying the original message (0: off, 2: use text/plain instead)
#
! report_safe 0


#   Set which networks or hosts are considered 'trusted' by your mail

Spamassassin likes to learn about its mistakes (both positive and negative). Since my users don't have local access to the system, I need to add aliases which allow people to forward mail attachments that are or are not spam so that Spamassassin can use that information in its learnings.
First step was to get the sa-wrapper.pl script from Stefan Jakobs. This script had a dependency on the perl modlue MIME::Tools which I used the following comand to download and install it (as well as a bunch of dependencies it had):

cpan -i MIME::Tools

Then I setup the aliases in /etc/aliases as follows:
```
# Spam training aliases
spam: "|/usr/local/bin/sal-wrapper.pl -L spam"
ham: "|/usr/local/bin/sal-wrapper.pl -L ham"
```
When I tested it, the script failed because it couldn't open/write to the log file. I manually created the log file and set it be writable by the tool.

The Switchover

The switchover had to be handled carefully in an attempt to not loose any mail as I moved things (or as little as possible). The sequence I worked out and used was as follows:

Stop mail services on both the old and the new servers -- ALL mail services: SMTP, POP3, IMAP, etc.
On the old server, tar up all of the existing user accounts and user mailboxes and transfer them to the new server.
Copy the /etc/passwd and /etc/shadow files to the new server and copy out the user accounts that are moving and add them to the existing /etc/passwd and /etc/shadow files on the new server.
Copy the /etc/postfix configuration files from the old server to the new server and merge in any of the local settings from the old server. In particular the virtual domains information for all of the domains I host had to be incorporated into the new setup.
Copy the /etc/aliases file from the old server to the new server editing the file to remove any extraneous/old/useless entries. Run newaliases to notify Postfix of the changes.
Untar the user accounts in /home on the new server and set the owner/group ownership as necessary.
Convert Mbox mailboxes to the new Maildir format on the new server.
While I do alot of relaying of mail, there are a number of people who actually get their mail off of my server and so I needed to move their incomming mail to the new server and beccause we changed from mbox format to Maildir format, I needed to split the mail up in to individual files.

I found a perl script to do the conversion (mb2md) which I downloaded from here. Ran a few tests and figured out that I would use the command as follows:

mb2md -s "full path to mbox file" -d "full path to Maildir directory"
And, since I was doing this as root, I would need to:
chown -R user.group "full path to Maildir directory"
so that the right user owned all the files.
Create Maildir structures for those users who didn't have mail in their mailboxes.
For those users who didn't have mail sitting in their mbox files on the old system, I would need to create the correct heirarchy within their login directory for Maildir delivery. So I ran a script similar to the following (I just did it from the command line, so I don't have an actual copy of the script) in /home:
```
for user in user_list
do
    mkdir $user/Maildir $user/Maildir/cur $user/Maildir/new $user/curdir/tmp
    chown -R $user $user/Maildir
done
```
On both servers: Edit the DNS records to change the IP address for mail.cahillfamily.com to be the new server and assign the name oldmail.cahillfamily.com to the old server. And, of course, pubish these changes.
Enable mail services on the new server (do not, for at least a day or so, enable mail services on the old server in order to force any mail in other SMTP queues to go to the new server).
Test the setup by sending emails to various users in my hosted domains from local clients, clients in my hame and from my work email account to ensure that the changes had propogated out to the real world.

Epilogue

That's about it... At least what I remember. I'm sure that there are things I did during the move that I forgot to write down, but I did try to record everything. I'll update this if/when I figure out anything I did wrong or forgot to note.

I hope someone out there finds this useful. I know I will the next time I need to move the mail server to a new system.

Tags : ubuntu / postfix / firewall / spamassassin / apache2 / ssl

Cool gadget #14

2009-03-03T08:40:00.000-08:00

I've always been an anti-multifunction office device kind of person. If you got a good printer, it sucked at scanning or faxing. If you got a good fax, it sucked at printing or scanning. If you wanted to print a lot inexpensively, you used a monochrome laser printer. If you wanted to print color, you used an ink jet type printer. None of the multifunction devices seemed to be good enough to replace multiple dedicated devices.

In my home office, I've had a good monochrome laserjet printer (HP 4000TN), a good inkjet printer (HP 1200DN), excellent fax machine (Xerox something or other), a good copier (again a Xerox something or other) and a decent scanner.

Well, that has finally changed. The quality of all-in-one devices has gotten good enough that I now find them acceptable for most office tasks. Well, I guess I should clarify that I find the higher end devices satisfactory. The low end devices still are missing or have brain dead implementations of many of the core features that I require. The HP Color Laserjet CM2320fxi Multifunction Printer is one such all-in-one printer. The cost is a bit high for some home purchases (I paid a discounted $850), but the functionality gives me all the magic features I needed and does them all well enough that I can get rid of the existing multiple devices I have lying about which, together, accomplish some of the same tasks.

This device does the following tasks very well:

Built-in network printing from any computer in the house.
Black/white laser printing
Color laser printing -- looks as good as anything I've gotten off inkjets
Automatic duplex printing (printing both sides of the paper).
Black/white copying (single or multi-page)
Color copying (single or multi-page)
Fax sending/receiving with auto document feed
Automatic Scanning to email of multi-page documents (PDF)
Print directly from camera memory cards

My only complaints are:

it is somewhat more noisy than my old laserjet printer, though after a few weeks I've gotten used to it and don't notice it all that much
it is tall (because of the scanner unit on top with space for paper outputs and with 2 input trays). So tall that I haven't hooked up the 2nd input tray or the top would hit the cabinets above. It would be nice if the scanner/control unit could be separated and placed to the side of the printer. Yeah that would look like two devices, but it would make it easier for my kids to see the top buttons.

Those are relatively minor nits. We are extremely happy with this printer and all of its features..

That said, I do continue to own a desktop flatbed photo scanner and a dedicated film scanner. I could probably do most of what I want to do with the flatbed scanner with the new device. However, there's a lot of convenience to having it on my desk easily reachable when scanning many prints and I can take it with me when I go to the parents house to scan old pictures there.

So while I have gotten rid of the fax machine, copier, laser printer and inkjet, I still have some specialized devices lying about. And, BTW, I sold the old devices for $100 and sent back the inkjet printer to HP for an upgrade rebate of another $100, so the net cost to me was just $650.

Tags : printer / HP / multifunction / CM2320fxi / all-in-one / gadget

Exercising on the road

2009-02-27T20:19:00.000-08:00

I've spent the past week bouncing up and down the west coast between San Jose, CA and Portland, OR -- not spending even 48 hours in either location at any point.

This threw a wrench into my exercise program because not only did I have to find the time to exercise, I also had to figure out what to do with my sweaty clothes when I checked out each day.

At first glance, you might think that's easy -- just put the wet clothes in one of the plastic laundry bags and pack it. That is what I typically do when I'm checking out on my way home. However, since I wanted to use the clothes to exercise each day and I didn't feel like putting on wet clothes to go work out, I needed to dry them out.

When I'm staying at the same place, I can just let them air dry and that works well enough. However, since I had to change hotels 3 times this week, I needed something else to do. I could have used the iron to heat up and steam them out, but it just felt like something was wrong with ironing sweat into my clothes.

I ended up using the room blow dryer to just blow them dry. Worked fine. Clothes were dry each day and nothing appeared to be growing on them (plus the rest of my clothes stayed dry.

In case you're wondering, I did an hour on the stationary bike each day. Not too shabby for an old man, if I must say so myself.

Tags : travel / exercise

Digitizing slides

2009-02-20T18:45:00.000-08:00

In the days of film cameras, one of the ways to take a lot of pictures cheaply was to use slide film rather than standard film. The film was around the same price, but when developing slides, you didn't get any prints, so rather than a $10 or $15 bill for the developing, the bill was just $3 or so (if I remember correctly -- in any case it was way cheaper). Others also would claim that the slide film was better for pictures & sharing since you could no project them to an audience (back in the days of 9 and 11 *inch* black and white TVs and *no* computers, there wasn't any other way to do it).

So, I have thousands of slides that I have taken over the years(several hundred from my honeymoon alone) and my mother-in-law brought over a bunch of slides that Angie's father had taken over the years (going back to the late 50s). I want to get all of these scanned into the computer so that we can share them and, if desired, print them.

Before I get into the nitty gritty, I want to lay out some ground rules that I have for scanning large batches of slides/negatives. These have grown out of my experience scanning film and your mileage may vary, but I think they are a good starting point for anybody thinking about a similar project. They include:

I want the process as automated as possible so that I can do real work while the scanning is going on. Processes that require manual intervention every few minutes means that I have to dedicate larges amounts of spare time that I just don't have (like any of you do).
I want "good enough" quality pictures to come out of the scan process so that I don't have to do any manual processing of the photos (other than rotating them). When I first started scanning negatives, I would do a raw scan at high resolution and then spend 15 to 20 minutes per photo to get them to a state where I liked them. This is clearly unacceptable for large amounts of photos.
So my model is to get them good enough off the scanner so that I can enjoy/share/watch/etc. without any manual processing.
I want to be able to easily figure out which slide/negative the photo came from after I'm done scanning in case there's a picture that I want to do more with (such as scanning at high resolution and lots of manual processing so we can print out an 8x10 or 16x20 photo). This means that I need to be able to figure out which negative from without having to resort to a manual search of thousands of slides.
I want to preserve the film in case someone wants to work with it years from now.
Speed is not the driving factor. Scanning thousands of slides/negatives will take time. What is key is that the work can be done while I'm doing other stuff. This leads to some choices on the scanning which actually make the scans take longer, but you get better quality scans and you get to keep working on the day job while you're doing the scanning.

These ground rules led to a number of choices I made in setting up this process. As I describe the process, I'll try to explain why and how I made these choices.

Choosing the scanner

The first issue to address is how am I going to scan slides themselves. There are two basic options for scanning slides:

Using the slide adaptor that comes with most flatbed photo scanners (if you have a multi-function device (otherwise known as an all-in-one), you're probably out of luck as they don't seem to come with options for scanning slides). These adapters typically require that you place some number of slides (typically 3 or 4) into the adapter, remove the typical white background for document scanning and then scan the slides).
I find this process painful for many reasons, the biggest one being that it's very time consuming and manual in nature. However, this isn't too bad if you don't have a bazillion slides to process.
Using a film scanner designed to scan slides and negatives (film) rather than scanning documents/photos. These typically do a much better job on film that the flatbed scanners and they usually also have substantial automation capabilities.

It just so happens that I have both types of scanners and for me the clear choice was to use the film scanner. My film scanner is a Nikon Super Coolscan 4000 ED (it's about 5 years old and has been superseded by the newer 5000ED).

Organizing for scanning

If you're like most people, your slides have not stayed in their little boxes that you get back from the developer and frequently they are intermingled (in some cases within one of those slide projector trays, in other cases in the little slide shoe box where you threw all the slides).

One note about handling slides: Most slides are raw film stored within a cardboard or plastic mount which just holds the film without providing any protection to the film itself. You should use care when handling the slides to keep fingerprints, water, dust, etc. off the slides. I recommend using low-cost lint free gloves available at most photo shops when handling the slides.

You can choose to stay with the disorganization and just scan things, or you can put the slides back into their original sets. I chose to do the latter because figuring out what's on slides and telling stories about them frequently his helped by the nearby slides on the same strip of film. Getting the slides back into the set and then perusing them in order helps greatly.

To get them back into sets, you need to look at each slide. Most slides, even those printed many years ago, will have two pieces of information on each slide. A slide number in one of the corners and a processing month/year stamp. Sometimes this information is printed on the slide. Sometimes it's embossed in the cardboard mount. In many cases, the printing is hard to read and you have to use some sleuthing to figure out what set the slide belongs to and what slide number it is in that set. In the slide below you can fairly clearly see the slide number (34), but the processing date (May 89) is embossed on the cardboard and a bit harder to see.

Once I had them all grouped in sets & ordered by slide number I simply rubber banded them and put them into my to-be-done box and then started cranking.

Scanning the slides

Setting up the scanner

My 4000ED has an optional slide feeder (SF-200) which can feed up to 50 slides at a time for automated processing. This is ideal for my project. However, in many of the reviews of the product and in various support web sites, I found that there were many complaints about slides jamming in the machine -- which would really interfere with my automatic process requirement. I came close to just blindly upgrading to the latest version of the feeder (SF-210) thinking that it had to be better than the one I already had. However, from the reviews that didn't seem to be the case.

I should note that after looking at the wide variety of slides that I had in my collection (especially when I added in the older slides from my mother-in-law) it isn't so surprising that this is an issue. The slides vary greatly in materials (plastic, cardboard, even some metal) and they varied greatly in thickness.

All that said, I found one suggestion in an Amazon review that recommended tilting the scanner about 10 degrees and instead of using the spring-loaded slide pusher, place a C battery into the tray (it would roll down with the slides adding just a small amount of continuous, even, pressure). I gave that solution a whirl and across about 2K slides only had 6 or so jams -- two of which were caused by material defects in the slide mounting (the film had curved out of the mount and caught on the next slide causing the two to load simultaneously). Not bad.

To accomplish this I used two index card packs to raise the one side of the scanner and just placed the battery into the tray as you can see below:

Setting up the scanner software

Nikon Scan 4 is the software package that comes with the scanner. I modified the default settings to enable the following features:

Enabled Digital ICE - which does a great job getting rid of dust and small scratches -- it's not perfect, but it does work pretty well.
Enabled Digital ROC and Digital GEM post processing - these do a level of fade & color correction that makes many scans presentable that otherwise wouldn't be without a lot of manual processing.
Enable multi-scanning 2x - each slide is scanned twice and the scanned data is averaged together -- this gets a better scanned picture on most slides.
Set resolution to 2,000 pixels/inch (about 1/2 the full res quality of the scanner) at 100% scale. Just to keep the pictures down to a reasonable size on disk and to make some of the post processing more efficient. I can always come back later if I want a better quality scan on a particular slide.
For each batch scan, I set the file name to a one up sequence starting with the year (so, for example, the slides I recently scanned had a base file name of si2009001 and a two digit sequential number of the slide within the slide set). When I processed the next batch, I would increase the base file name by one (e.g. si2009002). The net result is that I could tell which slide set and which slide within a slide set a digital file came from . For example, a digital file with the name si200904523.jpg came from slide 23 in the 45th slide set scanned in 2009.

Loading the slides

Emulsion side - Each slide has an emulsion side and a smooth side. The emulsion slide is the side that the image is recorded and it recorded backwards (to view the slide correctly you view through the slide from the non-emulsion side. This is important because most scanners will tell you that they want the emulsion side facing a particular way (either by directly mentioning the emulsion side, or by using pictures of a slide with an ABC on it (when ABC is backwards you are looking at the emulsion side). On most slides that have some kind of printing, the side that indicates "this side toward screen" or something like that is the emulsion side and the slide number and date stamp are typically on the viewing (non-emulsion) side.

Up vs down - the orientation of the slides (which edge is up) seems to be somewhat random with respect to the printing on the slides. In some cases they are both in sync (the slide correctly oriented when the number/time stamp are on the top. In other cases it's the opposite (the number/date stamp needs to be upside down on the bottom in order for the slide to be oriented correctly). I found I had to look at a few slides to figure out which way it worked with that set.

Landscape vs Portrait - while slides usually appear square, the film within the slide is not. When you're holding the camera horizontally (the normal position) the image will be recorded in a landscape mode (where the width of the image is longer than the height of the image). When you're holding the camera vertically (on its side) the image will be recorded in portrait mode (longer height, shorter width). This is important in slides because in most scanners you should not turn the slide to correctly orient the picture if it was taken in portrait mode. Just scan the picture in landscape mode and later, in software, rotate it 90 degrees to get it into portrait mode. The reason for this is that most scanners only scan the landscape portion of the slide and will miss some of the slide while recording some of the mount if you scan the slide in portrait mode.

Slide Numbers - most slide sets do not start with slide 1 (at least most of mine did not) and frequently that have slides missing (sometimes simply because the slide image was blank). I wanted the actual slide numbers to match the file names so I would start the file numbers with the first slide number and I would ensure that all slides were sequentially in order, filling in missing slides with slides from the end. When I had to do filling in, I would go back to the files after the set was scanned and manually renumber the fill-in slides to correctly represent their slide number.

Scanning the slides

I would simply load a set into the feeder (correctly oriented, emulsion side to the right when looking at the scanner) indicate in the software that I was feeding X slides and set the starting number at Y. Then I was off to do the real work while the scanner went along chugging through the slides in the feeder.

Slide Storage

In order to be able to quickly locate slides, as well as to provide for archival storage of the slides, I chose to use Print File Archival Slide Preserver sheets for the slides and placed a label on each sheet indicating the slide set (which was part of the digital file name) that the sheet contained:

You can get these at many photography supply stores. I purchased my at Archival USA.

Once I had the slides stored in the sheets, I placed the slide preserver sheets into Century Box Archival Storage Albums (that I also purchased from Archival USA). Another option would have been to buy the file hangers that Print File makes and simply hang the sheets in a file cabinet, but I preferred the storage box. Anyway, I placed the slide pages into the boxes and placed labels onto the boxes indicating which slide set ranges were in the box.

Miscellaneous Tidbits

Use the magnifying glass, Luke

I found having a magnifying glass quite useful in trying to determine the slide numbers and/or date stamps on slides as well as to try to determine the orientation of the slides on slides that had no markings. It was just plain useful. Get one and have it nearby when you're working on the slides.

Remounting Slides

In some cases, it might be worthwhile to remount slides. For example if the mount is damaged, too thick, or otherwise interferes with being able to scan the image. I had this with one particular set of slides that came from my mother-in-law. It seems that in the late 1950s in Europe, slides were mounted in metal mounts that sandwiched the film between two pieces of glass. When they got to me, they were in pretty sad shape:

So I ordered some slide mounts and peeled back the metal cover, separated out the film from the glass sandwich and mounted them into new slides which scanned much better than the originals had.

Summary

This process seems long and arduous, but in reality the most time consuming part (other than the remounting of that one metal set) was the organizing the slides step because many of the slides were mixed together, some had no writing on them whatsoever, many had slide numbers and date stamps that were almost unreadable (magnifying glass helped there sometimes).

Once the scanning got started, the process essentially amounted to about 5 to 7 minutes to swap slides and store the scanned slides every hour an a half or so (that's about how long it took to go through the average 30 or so slides per set with the settings I had used on the scanner software).

I'm very happy with most of the pictures and for those that I'm not happy with, the slide itself usually left a lot to be desired -- almost always because of low exposure on the film.

Tags : scanning / photograph / slide / 35MM / digital conversion / digitizing

Unsubscribing hell...

2009-02-18T03:57:00.000-08:00

For some unfathomable reason I decided today to try to unsubscribe to some of the various spam messages I get from reputable companies. I would never try to unsubscribe to the umpteen million messages I get about body parts enlargement (some of which wouldn't look so hot on my if they were enlarged) or performance enhancement as the act of unsubscribing just confirms that they have a real person on the other end of the email line.

So, for reputable companies in the US, they are required by the CAN-SPAM act of 2003 to have an opt out method in each email. From the FTC's web site:

It requires that your email give recipients an opt-out method. You must provide a return email address or another Internet-based response mechanism that allows a recipient to ask you not to send future email messages to that email address, and you must honor the requests. You may create a "menu" of choices to allow a recipient to opt out of certain types of messages, but you must include the option to end any commercial messages from the sender.

Any opt-out mechanism you offer must be able to process opt-out requests for at least 30 days after you send your commercial email. When you receive an opt-out request, the law gives you 10 business days to stop sending email to the requestor's email address. You cannot help another entity send email to that address, or have another entity send email on your behalf to that address. Finally, it's illegal for you to sell or transfer the email addresses of people who choose not to receive your email, even in the form of a mailing list, unless you transfer the addresses so another entity can comply with the law.

So, I took a look at several of my emails... The emails from Lands End, Sears, 1-800-Flowers.com, American Express and Apple all had links and they all worked as one would expect. The either directly unsubscribed you or brought you to a page that gave you a few options (different kinds of emails, change email address, etc.) and one or two clicks and you were done.

Microsoft, on the other hand, was a true royal pain in the *ss. I received an email from them that included the unsubscribe link at the top:

And another at the bottom:

So one would think that it's all kosher. That clicking on the link would get you unsubscribed. However, that wasn't to be the case. What you got when you used that link was a page which said that I had to use my Windows Live ID to manage my settings and that if I didn't have one, I would have to create a Windows Live ID account in order to manage my subscriptions.

So you can't just unsubscribe. You have to create an account on some Microsoft server.

Being the persistent one, I went ahead and did so. That required that I provide an email address and also required out-of-band email validation (where they send you an email that has a link you have to click on to prove that you actually have that email address.

Did that and got logged into Windows Live. However, all the stuff about managing my subscription was gone and there were no clear links on the page that would get me there. So I went back to the email that started this and selected the unsubscribe link again.

This brought me to the "Profile Center" where there was a link for manage subscriptions. I thought I was getting close, but no, there was another roadblock that they threw up. There was no email address in there (they didn't take the one I entered for my Windows Live ID account). So I had to enter it again. And, of course, before I could manage it I had to go through the email validation again.

Then back to the profile page and back to managing subscriptions where I could finally unsubscribe. Now I'm stuck with a Windows Live ID account that I don't want but I don't see any easy way to get rid of it.

I think this rigmarole they have set up is in clear violation of the spirit and intent of the CAN SPAM laws and should be fixed. I should be able to unsubscribe easily without having to create an account. I should be able to unsubscribe with a minimal of effort.

Kudos to Apple, Sears, and all the rest who, IMHO, got it right. Daggers to Microsoft who clearly got it totally and inexcusably wrong.

Tags : spam / Microsoft / can-spam / Live ID / identity / email

Digitizing life

2009-02-16T04:11:00.001-08:00

Like many people today, I have a large collection of analog media containing family memories. Much of it is my own, but a substantial portion belongs to either my or my wife's parents. This includes film negatives, slides, prints, video film, video tapes, etc.

The saddest part about this old stuff is that it deteriorates over time (even when aggressive archival storage methods are used). In addition, it's very hard to share and usually gets dispersed as various interested parties (i.e. siblings) request to take one of them (sometimes promising to make a copy and return the original -- and I'm sure some actually do that).

I have piles and piles of pretty much all of that other than video film. I have decided that it's about time to bring it all into the modern digital world and am digitizing all of it -- negatives from all the 35MM photos I took, prints from all of our kids class/sports photos or from those 4x6s that we don't have negatives for, thousands of slides (which, IMHO, were the old fashioned "digital" camera in that you just paid $3 to get the roll of film developed without any prints and then said you would print the photos you liked, but never got around to it :-)).

When I'm done, I expect to be able to share my entire digital collection with my family either directly or when I post the more interesting photos on Facebook :-). I also expect that when my kids grow up and leave the house, they will each be able to take a copy of our entire collection with them to be able to peruse whenever they like.

I'm going to write a series of blog entries describing what I've chosen to do for each type of media and how I proceeded. Hopefully some out there will find it useful in one way or another.

BTW - there are a number of services out there that will do this for you for a fee. I've chosen to do it all myself rather than use a service because I want to organize things as I convert and I want to have sensible conversions (if you used the video camera to record your kids birthday and your friends kids' school performance you don't want them on the same DVD -- at least I don't). I've also worked to automate the process as much as possible so I can do it while I'm doing other things.

Finally, I've accepted that this will take a long time and not be done overnight and I will methodically work through the piles (and they are large piles).

Wish me luck!

Tags : digitizing / scanning / film / 35mm / slides

Another change in United's Mileage Plus Program

2009-01-18T11:32:00.000-08:00

I've written about some of the changes United made to the Mileage Plus program for 2009 (most of which I don't like), but I just noticed one that hasn't been documented much of anywhere that I have found. Kind of just snuck in there.

In the past, when you qualified for one of the premier levels, that status was good through the end of February the following year (so, my 2008 1K status was good through the end of Feb 2009). However, for my 2009 1K status my card is only good through the end of January 2010 -- a month shorter.

This is probably because with all the electronic record keeping, they think they don't need the extra month to get all the records in order to determine status.

This was not mentioned in the 2009 program changes page.

We'll see if my Global Services card (which I haven't received yet, half way through the month) has the same timeframe or if it gets the extra month when my card, hopefully, shows up.

Tags : United / ual / Mileage Plus

United comes clean on Global Services status

2009-01-06T07:32:00.000-08:00

For years, United has been very secretive about how one becomes a Global Services member. My blog entry on Global Services is still the most popular page here, 2 years after it has been written -- accounting for more than 25% of my page hits. It's even the number one search result for "United Global Services" on Google (yeah, I'm proud :-) ).

When I logged into my united account today, I found the following published on the web site:

Ensure your Global Services status for 2009 Fly 50,000 miles on United® or United Express® in First (F, A, P), Business (C, D, Z), or full-fare United Economy® (Y or B) during 2008, and your Global ServicesSM membership will be renewed for the 2009 program year.

Track your progress by visiting united.com/gstracking.

Of course, it's kind of late for the 2009 program year at this point. But still it's now out in the open as to what you need to do to qualify. They even have a web page that you can go to to check your earnings status. Mine is still showing my 2008 earnings (since I know I have absolutely zero earnings in 2009).

Of course, I'm not convinced that this is the only way to get Global Services status. I think that their marketing and business relationship department will use GS stats as a reward for important business partners who bring them substantial corporate business, even if they, themselves, don't fly a lot. That's business as I would expect it to be.

Tags : ual / United Airlines / Global Services / travel / Mileage Plus

Situational Awareness

2008-12-19T09:54:00.001-08:00

One of the best defenses against phishing, scamming or pretty much any other type of social engineering attack is to be aware of your situation and what to expect to have happen as well as to know when it should happen. The various attacks that come along should all raise red flags at several steps in the process. In the real world, we get this through millions of years of survival training -- those who didn't sense trouble usually died out before they could reproduce.

However, in the internet world, most of the visual and/or aural queues that raise your sense of awareness and caution are missing and we need to learn a new set of such protection mechanisms.

To that end, I'm going to periodically talk through an attack and point out things that one might notice which should cause you to think twice about continuing (or at least do a much more detailed check of whats going on before you continue).

Today, I received an interesting email reportedly from "Classmates.com" (which, of course, we all know we can't trust as anyone can claim to be anyone else with current mailing technologies):

Your Classmates Events: Reunion January 16th 2009 " With pride and joy we invite you to share a special day in our lives and join us for the Class Reunion on Friday, January 16th 2009. Bring the gang from Our High School back together again! Great party - from start to finish! " Proceed to view details: http://video.classmates.completeserv.user-v5mn1ckah.newyearclassmates.com/messages.htm?/type/INVITATION=m5kibxmz390kynf Your favorite people are already here, so use ClassmatesTM to bring them together. With best regards, Carmine Hilton. Customer Service Department. Copyright 1995-2008 Classmates Online, Inc. All Rights Reserved.

At first glance this seemed somewhat legit because I am a member of Classmates.com and so could reasonably expect to get emails from them. I'm also in a graduating class that would have an interesting anniversary in 2009 so it does make sense that we would be scheduling a reunion.

However, the email address to which the email was addressed is not the one that I have associated with classmates.com account - so clearly it wasn't classmates.com sending me the email. The address that was used is one that I've had for ages and typically gets close to 99.9% spam, so my internal "what's going on here" guard sprung up.

In addition, the email didn't look like the typical Classmates.com email -- which is just stupid laziness on the part of the attacker as it's pretty easy to fake someone else's email style, so while the email looking right isn't a good sign, having it look wrong is a big red flag.

Finally, the link in the email wasn't at the classmates.com domain (to find the actual domain you have to look at the third slash (/) in the URL and then work backwords -- the first two slashes should be right after the http: at the begining of the URL, so it's the next /). In this case it was newyearclassmates.com which should be another big red flag since it clearly was made to look like the real classmates.com domain.

If you did, somehow, follow the link, it brought up the following page:

This, too, doesn't look like the Classmates.com site -- another red flag and has no real information about what's going on. One would expect to at least have some text at this point with the name of the high school and other such information.

Instead all you have is a thing that looks like a video player application but actually is just an image and if you click anywhere on the image (like the play button or, if you're thinking of a YouTube video, the center of the video image) or on the Adobe Get media player button, the site tries to download and run a native application (an EXE). That should send big "DANGER WILL ROBINSON" shivers up your spine. Any website that tries to download an exe directly to your platform has to be treated as the enemy until proven to be a friend (no innocent until proven guilty here -- good sites rarely download EXEs directly like that without at least having some interactions with the user).

In this case the executable was Adobe_Player10.exe -- which I'm sure is a Trojan Horse which would do very nasty things to your computer at some point and it wasn't coming from Adobe's own web site, but rather from the newclassmates.com site itself -- another red flag (which, I hope, you never got because you didn't get to this stage). If you did get here and you think everything's legit, you should stop, go to the adobe web site and check version numbers or at least download the application directly from Adobe -- never download/install software that you got to through an untrusted link or from an untrusted site.

UPDATE: I've gotten 7 more of these same invites. All to different email addresses that route to me. That's another really good sign that things aren't well in Kansas and you should stay away from the email.

Moral of the story: It's a jungle out there and you've gotta watch out for yourself as there's nobody else doing it for you.

Tags : hacking / phishing / Trojan Horse / Internet Self Defense

Facebook vs DNS

2008-12-03T05:21:00.000-08:00

Sometime back, about a couple of weeks ago, my Facebook page loads all of a sudden started getting very slow (like 20 seconds or so before the data started loading, but once it did start loading it was fast). It was only happening at Facebook (Google, WheresGeorge, Blogger, pretty much any other site) was working fine, so I thought the problem had to be at Facebook rather than on my side.

However, after it kept up for a week, I started to get irritated enough to dig into it. First I turned off my web proxy and went directly to the sites from my browser. Things worked fine then, so clearly it was an issue in my proxy. I run a Fedora Linux server at home that serves as my web proxy using the Apache HTTP daemon.

This past weekend, I started digging into the problem and spent several hours debugging, testing, searching the web and while I still don't have a clear reason as to the why, I do understand the what and have put together a somewhat nasty hack around the problem. Hopefully I will dig around and find or figure out what the problem is so that I can put in a good fix.

My first look at the server didn't show anything amiss. The httpd logs showed the accesses to Facebook with no errors. That led me to consider DNS as this felt like what you get when your DNS is timing out.

My /etc/resolv.conf file was clean and correct. Using the nslookup or dig tools, I was able to look up the names without problems and quite quickly on both my own name server as well as the name servers provided by my ISP. The system logs didn't show any problems in named or anything that looked like the firewall could be getting in the way.

However, using any other tool (telnet, wget, httpd) the name look ups would go through several failures before succeeding -- causing a substantial delay in accessing the site. This only happened with Facebook related sites (www.facebook.com and apps.facebook.com to mention two of them). The same tools, accessing any other site that I tried, had no problems and no delays.

Using strace, I could see that the first pass at the name service look ups were failing and each timing out after so many seconds before trying the next. Eventually, the tools go back and try again and the second time, the response comes back almost immediately and the tool continues. For example, "wget http://www.facebook.com" returned the following:

01     0.000106 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
02     0.000068 connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.1")}, 28) = 0
03     0.000076 fcntl64(3, F_GETFL)       = 0x2 (flags O_RDWR)
04     0.000054 fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK) = 0
05     0.000042 gettimeofday({1227974358, 62163}, NULL) = 0
06     0.000048 poll([{fd=3, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
07     0.000059 send(3, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
08     0.000861 poll([{fd=3, events=POLLIN}], 1, 5000) = 0
09     4.998266 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4
10     0.000065 connect(4, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.36.226.50")}, 28) = 0
11     0.000071 fcntl64(4, F_GETFL)       = 0x2 (flags O_RDWR)
12     0.000046 fcntl64(4, F_SETFL, O_RDWR|O_NONBLOCK) = 0
13     0.000041 gettimeofday({1227974363, 61621}, NULL) = 0
14     0.000046 poll([{fd=4, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
15     0.000053 send(4, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
16     0.000098 poll([{fd=4, events=POLLIN}], 1, 3000) = 0
17     2.998500 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 5
18     0.000070 connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("207.228.225.50")}, 28) = 0
19     0.000073 fcntl64(5, F_GETFL)       = 0x2 (flags O_RDWR)
20     0.000045 fcntl64(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
21     0.000043 gettimeofday({1227974366, 60548}, NULL) = 0
22     0.000045 poll([{fd=5, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
23     0.000052 send(5, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
24     0.000118 poll([{fd=5, events=POLLIN}], 1, 6000) = 0
25     5.997342 gettimeofday({1227974372, 58108}, NULL) = 0
26     0.000050 poll([{fd=3, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
27     0.000054 send(3, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
28     0.000416 poll([{fd=3, events=POLLIN}], 1, 5000) = 0
29     4.997778 gettimeofday({1227974377, 56418}, NULL) = 0
30     0.000063 poll([{fd=4, events=POLLOUT, revents=POLLOUT}], 1, 0) = 1
31     0.000055 send(4, "\0079\1\0\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 34, MSG_NOSIGNAL) = 34
32     0.000106 poll([{fd=4, events=POLLIN, revents=POLLIN}], 1, 3000) = 1
33     0.001235 ioctl(4, FIONREAD, [34])  = 0
34     0.000065 recvfrom(4, "\0079\201\202\0\1\0\0\0\0\0\0\3www\10facebook\3com\0\0\34"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.36.226.50")}, [16]) = 34

As you can see, the delays come waiting for a response from the nameserver and it's not until the second try on the second name server (lines 31-34 before we get a response. You might think that this has something to do with my name server on 127.0.0.1, but that wasn't originally in my /etc/resolv.conf file until I started the debugging and the problem still occurs when I remove it.

A similar trace of the dig command shows that the first name server (whether it be 127.0.0.1 or my ISPs) resolves the name almost immediately (though dig uses a different communications method (sendmsg vs send) and different networking libraries.

Traces for wget with other host names return successfully on the first lookup.

I haven't (yet) figured out what exactly is causing this. But I have figured out two workarounds (neither of which are all that nice):

Set one of Facebook's name servers as the first name server in my resolv.conf file (so my applications use that name server to resolve all host names.
This does work (name resolutions worked first try and in very reasonable times). However, name servers are core trusted parties in your network access and I really don't like setting things up so that I totally trust Facebook's server for all of my outgoing name service look ups. Call me paranoid, but this one just isn't right for me.
Add www.facebook.com and apps.facebook.com host entries to my /etc/hosts file (which is checked before name service look ups.
This definitely works, though it does remove the usefulness of DNS from my access to Facebook (like if they change their IP address I won't know). However, it is the lesser evil of the two solutions I have found so far and so this is what I've done for now.

I'll post an update if I figure out exactly what's wrong (which I'm very unhappy about not being able to figure out so far -- I like being able to understand things and spent several hours after I had workarounds trying to figure it out to no avail).

Tags : facebook / dns / name server

Paul can't be wrong all the time

2008-12-03T03:32:00.000-08:00

I have to say that, for once, I totally agree with Paul. In responding to a post by Ben Laurie, Paul disagrees with Ben's opinions of passwords and phishing.

Ben had said (and I'm showing a bit more here than Paul did in his response):

Well, no. If your password is unphishable, then it is obviously the case that it can be the same everywhere. Or it wouldn’t be unphishable. The only reason you need a password for each site is because we’re too lame to fix the real problem. Passwords scale just fine. If it wasn’t for those pesky users (that we trained to do the wrong thing), that is.

First off the phishability and reusability of passwords are distinct and separate issues. They have pretty much nothing to do with each other.

The primary reason one should not use the same password everywhere is that once that password is discovered at one location, then it can be reused at other locations. So, if, for example, you use the same password at Amazon, eBay, PayPal and Facebook, all one needs to do is find out your password on Facebook and then they will be able to sell things in your name on eBay, buy things in your name using PayPal and ship lots of things in your name at Amazon).

As Paul mentioned, there are many attacks to finding your password -- an administrator at Facebook could look it up in the password database, you could have a weak password that the hacker could attack via brute force (and if you're using the same password everywhere, they could use multiple sites to break the password making all/most of the anti-brute force rate limiting capabilities at a given site pretty moot). Just to name a few.

All of that said, Ben did have several good points in his post. Yes, we, as an industry, have done a terrible job in the usability of passwords. The typical user has been prompted for passwords so often and in so many places that they have no feel for when it should or shouldn't happen (one of the best personal defenses against phishing).

Personally, I think the utopia for online identity comes in with strong authentication to a small number of identity providers which assert my identity through SSO and Federation out to a large number of relying parties. Ben's point about the attacks around issuance/re-issuance of such strong credentials is very valid -- they can't be based on much weaker socially engineerable factors. The credentials will end up having to be issued with strong levels of assurance.

I also look forward to being able to login once at the start of my day and maintain that state in a reasonably secure fashion for the entire day without having to re-authenticate every few minutes or deal with "your session has been terminated for your security" when I've been sitting at the computer the entire time.

Tags : identity / authentication / assurance / federation / SSO / password / phishing

Is Sir Bonar one of Paul's aliases?

2008-11-18T12:29:00.000-08:00

I just have to say that the article on ContactPoint written by Sir Bonar and quoted by Kim just feels like it was written by our one and only Paul.

Either Paul is writing under an alias, someone is working hard to emulate his ironic style, or somebody is writing seriously and just doesn't have an f***ing clue.

Interesting, very interesting....

Tags : identity / identity theft / security

Delayed Upgrades

2008-11-13T12:44:00.000-08:00

One of the benefits one gets for being an elite member of United's Mileage Plus program is the ability to upgrade into the next class of service on select fares (most domestic fares qualify and some international fares qualify). Theoretically, there's also a benefit to being at a higher level in the program as your upgrades should clear sooner:

Status Clears at

General Member 24 hours before flight

Premier Associate 36 hours before flight

Premier 48 hours before flight

Premier Executive 72 hours before flight

Premier 1K 100 hours before flight

Global Services 120 hours before flight

Status	Clears at
General Member	24 hours before flight
Premier Associate	36 hours before flight
Premier	48 hours before flight
Premier Executive	72 hours before flight
Premier 1K	100 hours before flight
Global Services	120 hours before flight

This used to work pretty much dependably until there were very limited seats left (the last one or two seats usually were left until boarding time).

However, this fall I've noticed that United has not been clearing upgrades, even when there are a multitude of seats available. For example, I'm on a flight tomorrow (in less than 24 hours) that has 8 of 12 seats still available for purchase but my (and presumably several other's) upgrade still hasn't cleared.

This has been pretty consistent on the last 8 or 10 flights I've been on, both domestic and international. It seems that the guys in "inventory control" (the part of United that makes the seats available for upgrade) has decided to not release any seats for upgrade until 10-12 hours before the flight.

This kind of makes the cool table of when things clear pretty useless and, to some extent, a bit of misleading marketing if not an outright lie.

Here's to hoping it's just a temporary glitch in their systems and things will get back to normal soon.

Tags : United / ual / Mileage Plus / upgrade

Paying for upgrades

2008-11-07T07:08:00.000-08:00

United Airlines has announced a host of changes for their Mileage Plus program for 2009. Many of the changes involved increased mileage for award travel (other than domestic economy travel).

However, the worst change, IMHO, is that like American Airlines, United is now going to charge $$ (in addition to mileage) for mileage based upgrades from anything other than full fare economy tickets.

To me, a long term, very loyal 1K, million mile flyer, this really sucks. This was the one real benefit (upgrades without $$) that would drive business travelers to want to fly on the same airline. Now our business trips are going to cost as much as $1,000 if we want to upgrade both directions on an international flight.

United, I suggest you reconsider this change or, a bit selfishly, make an exception for your most loyal customers (1Ks/GSs) like you do for most other fees. Otherwise, I suggest that those of you who are flying in 2009 or early 2010 make your upgrade requests prior to July 1, 2009 (the effective date for the upgrade charges).

I also suggest that if this change bothers you, you take the time to let United know so. Recently, negative feedback about moving to pay for meals on international flights cause United to change their minds and maintain their current meal program on such flights. Perhaps we can do the same with upgrade charges.

Tags : travel / ual

Data Privacy Day

2008-10-02T08:49:00.000-07:00

Please join the US, Canada (yeah, it's not just a blue state), and 27 European countries in celebrating second annual Data Privacy Day on January 28, 2009.

Designed to raise awareness and generate discussion about data privacy practices and rights, Data Privacy Day activities in the United States have included privacy professionals, corporations, government officials, and representatives, academics, and students across the country.

One of the primary goals of Data Privacy Day is to promote privacy awareness and education among teens across the United States. Data Privacy Day also serves the important purpose of furthering international collaboration and cooperation around privacy issues.

You can get more information, presentations, event information, etc from the Data Privacy Day web site.

Join the Facebook Data Privacy Day 2009 Group to hang with other participants and follow along with the developments.

Tags : identity / privacy

Changing Planes

2008-10-01T05:15:00.000-07:00

It happens to me a lot more than I would like. I'm booked on an Airbus A320 only to have United change it to an Airbus 319 causing my exit row seat in row 11 to be a standard economy seat (not even an economy plus seat) -- that's why I'm not too keen on booking exit row seats nowadays -- though booking exit row seats is one of the primo perks of a United Mileage Plus Premier Executive.

However, it seems to be a much worse change when you've got a seat booked in United's new Premium International Class only to have United change the plane at the last moment and replace it with a standard configuration plane. This happened to me 3 out of 4 flights this summer between Dulles and Frankfurt.

I mean would you rather have this (the old confirguration):

Or this (the new configuration):

It felt like a big bait-and-switch to me. Show me the cool fancy new seats that are a world of difference better than the standard seats (the premium seats lie flat, have 15" screens with 100s of video on demand shows/movies, have cushy cushions, etc., etc.) and then stick me in a standard configuration without telling me till I get on the plane. No notice before hand. No chance to change to a different flight. No compensation whatsoever. Not even an "I'm sorry."

I could understand this if it happens once in a while, but 3 out of 4 flights doesn't sound like once in a while. I could also understand it more if there wasn't such a big financial benefit to United in using the standard configuration plane (they get to sell a whole lot more business and first class seats in the old configuration than in the new configuration). How do I know that United isn't simply saying "well, we've oversold business by 20%, so let's use the standard configuration plan so that we can scoop all that revenue." ? There's also the fact that United started publicly announcing that they were using the new configuration planes on Asian international routes around that time, so perhaps they moved the planes from the europ

Perhaps I should take the advice I received from my friend George (who was on the last such change with me): Just go with the flow and be happy with what life brings you. That would certainly be better for my blood pressure, but I just don't think that's me. I think United should offer some form of compensation to those who chose to fly on the plane because of the premium seating that United is heavily advertising.

I guess the only thing to learn from this experience is to not depend upon the new configuration planes until United has completed its roll out of the upgrades. Originally the conversion was to be complete in 2009, but now they are predicting 2010. So far, as of Sept 2008, they have only converted 13% of their international planes (7 of 21 767s, 5 of 24 of 747s and 0 of 46 777s).

Tags : United Airlines / ual / travel / Dulles / Airbus / A320 / A319

Smart Card hackery

2008-09-30T11:16:00.000-07:00

This is an old video (from May of '08) and probably accomplished using an older technology smart card (theoretically easier to break), but it's still quite interesting to watch how one can peel back the layers of a smart card in order to snoop the communications going on within the components.

The related story on Wired.com gives a lot of interesting details to the ongoing cold-ware between satellite TV operators and hackers attempting to get free TV.

Tags : smart card / hacking

Cardspace, Liberty, & Intel's ICP

2008-09-25T03:57:00.000-07:00

A couple of weeks back at DIDW 2008, I reported on a proof-of-concept that we put together at Intel where we combined Cardspace with our Identity Capable Platform (ICP) to show how ICP could extend/strengthen a cardspace deployment. While we used Cardspace in this demonstration, the code should work with any Identity Selector conforming to the Identity Selector Interoperability Profile.

For those of you who don't know, ICP is a research project we have been working on at Intel exploring how identity capabilities could be added to a platform to enhance online transactions. Our contributions to the Liberty Alliance's Advanced Client Technologies are part of that work.

In this proof-of-concept, we showed how a mythical bank (ACME Bank, of course) could provision an identity agent to the platform which was then subsequently used as the identity source for Cardspace when the user initiated a session at the bank. To Cardspace, the identity agent was a full fledged STS and had a managed card that has been provisioned into Cardspace (so, essentially, this was an off-the-shelf Cardspace deployment).

The provisioning process made extensive use of the Liberty Advanced Client Technologies protocols to securely provision the identity agent to the platform.

One might ask what exactly is an identity agent. I use the term very loosely to define any identity related agent software. In this particular case, the identity agent exposes WS-Trust and ID-WSF Provisioned Module interfaces as well as containing a SAML token generator and an ID-WSF IdP Service client (to be able to get minting assertions).

If you want to take a look at the presentation it's here. However, I have to warn you I write my presentations as something that needs speaking to and not as standalone documents.

Even better, there's going to be an encore presentation as a Liberty webcast on November 18th. I'll post the details once I get them.

UPDATE: Britta found it for me: Info/Registration for Webcast . Where would we be without Britta!

Tags : identity / Intel / Liberty / Cardspace / WS-Trust / SAML / Identity Selector / DIDW

Absentee Ballots

2008-09-22T12:01:00.000-07:00

At last week's Liberty TEG F2F in Boston, Hubert (the guy living in French alps who just recently became a US Citizen) pointed out to the rest of us that the fall Liberty Alliance Sponsor's meeting in Tokyo is taking place the week of our presidential elections here in the US.

So, those many of you who will be attending the meeting in person should head on down to your local registrar (or however you would do it within your state/county) and register for an absentee ballot.

In Virginia, they only allow absentee voting for a limited set of reasons, none of which include "I'm more comfortable voting from home" or "I don't want to have to deal with the long lines at the local precinct." I think that they should allow anybody to use an absentee ballot, regardless of reason (even if they just feel like it). I mean, that's the point, isn't it: Get the person's vote counted.

I also don't like the fact that some/many/all places that use absentee ballots, only count them when they can make a material difference in the outcome (e.g. if the election's difference in votes is less than the total number of absentee ballots). I think that sucks. I would rather they just always count them (and perhaps start with those numbers first. Just makes sense to always count a vote. Imagine if they chose to not count a state's votes if the state's population couldn't make the difference in the outcome of a race.

In any case, if you're going to the meeting, be sure to get your ballot. This is sure to be an interesting election (though I wouldn't mind an Obama landslide -- even if that meant that they didn't count my absentee ballot).

Tags : voting / Liberty Alliance / absentee / absentee ballot / Obama

What ID-TBD means to me....

2008-09-17T08:47:00.000-07:00

For those that don't know what ID-TBD is, it's an effort underway trying to tie the umpteen different identity efforts together into an uber identity organization. TBD as in To Be Determined (as in, we don't want to argue over the name till we get agreement on the organization and organizational structure).

My main goal here is to get out of the Liberty Alliance and away from it's exotic meeting locations like Singapore, Paris, Stockholm, Tokyo, Madrid, Sydney, Rome, etc.. I have become an active member in the Liberty 50 (those of us who have put on an extra 50 pounds or more since starting to participate in the organization). I'm probably at the head of the line and perhaps hit my peak at around 60lbs (30 or so kilos for the rest of you guys outside the US).

Yes, I blame Liberty for this (not my lack of good eating habits, my desire to have hamburgers and fries for every mean -- even breakfast -- my lack of exercise, etc., etc.). It's clearly Liberty's fault. You can see it in the pictures below:

That's me in 2001, shortly before I joined Liberty. And now, after 7 years participating in Liberty:

So by exiting Liberty and joining ID-TBD, I hope/expect to be able to loose my Liberty 50 and go back to my 2001 self. Even with just the announcement of the potential organization, I've made some progress in that direction:

This is why I am sooo supportive of the new organization. It has nothing to do with messaging convergence, coordination, consolidation or any other such mom and apple pie reason for me. I just want to get out of the Liberty 50 group!

Tags : liberty / ID-TBD

Let me count the ways

2008-09-16T05:44:00.000-07:00

Washington Dulles airport now has 4 separate security checkpoints for non-employees. These include:

Regular security checkpoint. This is the old tried and true security queue on the check-in level of the airport. These are intended for use by the average traveler and frequently, especially around 4PM, has long, slow moving lines.
Premium security checkpoint. This checkpoint is co-located with the regular security checkpoint but it has its own dedicated queue. This queue is restricted to premium travelers (those in first/business class or those traveling on a flight where they have premium status -- such as United's Mileage Plus Premier members). This queue is typically much shorter and sometimes moves faster than the regular security queue. Dulles added premium security lines a couple of years ago.
Registered Traveler (Clear) security checkpoint. This checkpoint is restricted to people who have paid the annual $120 fee and subjected themselves to a background check. The registered traveler checkpoint at Dulles is managed by Clear. This checkpoint is down on the arrivals level near baggage claim 8 and is shared with the Employee checkpoint. Very short lines, quick processing (other than the time the x-ray scanner got a bag stuck in it with mine in there as well).
Dulles Diamond security checkpoint. This is a new checkpoint that just recently opened on the arrivals level near baggage claim 7. The signs for this checkpoint say it is only for expert travelers (2 trips/month) traveling alone, with only one carry on item and all their liquids already in bags. Theoretically these frequent travelers know what they are doing and the line can move along at a good clip. I tried this checkpoint on my trip up to Boston yesterday. There was no verification that I was a frequent traveler (though if they've read my blog, they will know). I think any single traveler could walk in there. I also verified that you can go through with a carry-on bag and computer bag (the sign says only 1 carry on item so I thought they might be restricting those of us who also bring along computer bags). So it would seem that anyone traveling alone could use this queue (and it was totally empty when I came through mid-day). Perhaps they will have tighter checks when the queue backs up once people notice it is here.

Tags : travel / Dulles / United Airlines / Registered Traveler

Slamming SAML..... NOT!

2008-09-15T13:27:00.000-07:00

Jeff responds to my note earlier suggesting that using psudonymous identifiers adds security depth:

This is a very dangerous suggest as it implies that SAML is not secure enough without pseudonymous identifiers, the use of which makes SAML deployment a lot more complicated. Pseudonymous IDs are for privacy not security. If your system requires them to be secure, you have done something wrong. Period.

I was in no way suggesting that SAML was not secure enough. However, I am of the opinion that any SSO system (including SAML) is weaker, from a security and a privacy point of view, without pseudonyms than the same system would be if it was using pseudonyms. That doesn't say or imply that it isn't secure without them, just that it would be better with them.

And I stand by my statement that had Google used good pseudonyms across relying parties, the impact of their lack of the audience restriction would have been minimal. That isn't saying that I think a system should rely on pseudonyms as their primary security model, just that the effect would have severely reduced the impact of the error.

Tags : identity / SAML

Pseudonymity would help

2008-09-15T03:00:00.000-07:00

Kim Cameron writes of Google's failing to scope SAML assertions:

But according to the research done by the paper’s authors, the Google engineers “simplified” the protocol, perhaps hoping to make it “more efficient”? So they dropped the whole ID and scope “thing” out of the assertion. All that was signed was the client’s identity.

The result was that the relying party had no idea if the assertion was minted for it or for some other relying party. It was one-for-all and all-for-one at Google.

While I agree totally that the intended recipient should have been identified within an <AudienceRestriction> in the SAML assertion (how SAML shows the intended scope of the assertion) the problem would have been moot if Google used good pseudonymous identifiers for its users.

Pseudonymous identifiers are random identifiers that change for each relying party (so my identity at relying party A might be 123 while my identity at relying party B might be 345). Good pseudonymous identifiers are large random values (so that they are unpredictable) and are not reused across multiple users (so the same identifier is never used at different relying parties for the same or different users).

The primary impetus behind pseudonymous identifiers is to prevent the use of the identifier as a correlation factor across multiple relying parties -- in contrast, a globally unique identifier would allow relying party A to ask relying party B about what user 123 did yesterday, whether or not the user was around. However, pseudonymous identifiers also provide the following benefits:

added security depth - an unknown user identifier adds another layer of security on the SSO system (which, in this case, would have protected the user accounts from attack since even if the assertion went to a different relying party, there would be no user account with that specific identifier, so it wouldn't be useful).
easier integration of new partners - when integrating new partners, the identity systems of the partners may have different data structures for user identity (at it's most simplest case a new relying party may store user identifiers in 32 bit integer values, while the IdP typically uses 128 bit random values -- a system that supports good pseudonymous identifiers and the assumption that identifiers are different on each system will easily be able to handle this.

One might be concerned about how relying party A could invoke a service of relying party B when they are all using different identifiers (such as a google relying party using Google Checkout). This is pretty simple. Typically, any such service invocation requires relying party A to get a security token for the user at relying party B. When that token is obtained, the issuer does the identity translation. SAML provides for the protection of the identifier in the assertion using encryption since relying party A should never know what the user's identifier is at relying party B and the assertion is given to relying party A.

Liberty ID-WSF provides several entities that provide this translation services depending upon the topography of the deployment. The most common such service is the ID-WSF Discovery Service.

Similarly, in WS-*, the WS-Federation Pseudonym service is called out to perform the same translation service (and it is possible for a deployment of a WS-Trust STS to perform this translation internally during token generation).

I strongly recommend that any deployment of SSO, even within a single enterprise, make use of pseudonymous identifiers. They only strengthen the identity infrastructure.

Tags : identity / Liberty / ID-WSF / ws-federation / google / SAML / pseudonymous identifier

Paul, Paul, Paul....

2008-09-12T07:52:00.000-07:00

Paul writes about an upcoming Liberty Alliance futsal match in Tokyo and includes:

Conor "One-Sock" Cahill, when asked whether he would be participating, responded 'Only if I can get an upgrade to First. Currently, I'm booked in business on a Triple 7 in from SFO, but I'm trying to switch that because I'm in seat 4A and I hate that seat because the power plug is about 2 inches too high and I have to unbuckle my seatbelt to reach it. I generally like 3F but the window shade was broken last time and the sun woke me up, even though I had taken my Ambien.'

Paul, everyone knows that there's no 4A on a United Airlines Boeing 777. First class stops at row 3 and business class starts at row 8. 3F isn't a window seat (3A and 3J are, though 3A is frequently reserved as a pilot rest seat, but not on the long haul triple 7 that United uses for IAD->NRT flights).

My preferred seat is, of course, 3A since they would have to pick the best, quietest seat for pilot rest, followed by it's opposite window seat 3J.

And finally Paul, you *know* that I don't sleep on the way to Tokyo. My rule for flying west is to stay awake till arrival at the hotel. In fact, I remember you telling me that you had tried doing the same and it worked for you as well.

Please try to get your facts somewhat correct when generating a fake Conor "Mr. Travel" Cahill quote. And stop ragging on me about the sock. I was in the middle of putting my shoes on when the called us together to take the picture.

Tags : futsal / Liberty / United / UAL / Boeing / 777

Lipstick & Pigs

2008-09-11T18:15:00.000-07:00

Just to be clear about lipstick and pigs, I want to point out that during my DIDW presentation -- before anybody questioned it -- I pointed out that our proof-of-concept demo showed a strong authentication credential being issued based solely on a username and password. I also explained that in a real-world situation, the bank would have only issued the credential under some higher level of authentication and went on to describe several options the bank would use.

Paul "claims" he had picked up on that issue before I mentioned it. With him sitting next to the very distracting Pamela during the session, I'm not sure we should believe him.

Close Friends

2008-09-11T15:17:00.000-07:00

I've recently become active on Facebook, reaching out to a number of people with home I have worked/played/lived or otherwise come across over the past few years. I know it shocks Paul that I actually seem to have some people whom have confirmed that I am their friend (including Paul himself).

Facebook allows me to define access to portions of my profile depending upon a users status:

Friend - someone with whom I have a direct relationship (they're in my list of friends)
Friend of friend - someone who has a relationship with someone that I have a relationship with
Network - A group of people that is organized based upon geographic locations, work, etc. I belong to both the Intel and Washington DC networks.
Public - everyone else

This seems to be a simplistic picture of the world of relationships. I can see how I would like to be able to classify some people as acquaintances, some as friends, and some as close friends (giving them different access to my profile information). Just like I bring friends and close friends to my house, but not usually acquaintances.

This came up when, out of the blue, I received an friendship request from someone who I didn't know at all, but they were interested in one of the groups I'm interested in (solar energy) and it probably didn't hurt that they happen to be a fairly nice looking example of a female member of the human race. I wouldn't mind allowing them in as an acquaintance, but I really don't consider them a friend, nor do I want them to be able to see some of the portions of my profile that I make visible to friends. Also, as a responsible friend of my friends, I wouldn't want her to get access to the information exposed by my friends to friends of their friends (Paul, if you can't follow that, I can draw you a ven diagram of it later).

So I'd like to see some extended attributes around relationships added to social networking. Not just at Facebook, but also at other sites like Linked-In.

Tags : Facebook / Intel / relationships / social networking / Linked In

Identity Leakage

2008-09-09T05:49:00.000-07:00

It's interesting to see how much information you can learn about people just sitting around at the airport.

This past Sunday, I flew out of Dulles airport and running a bit late I arrived just 45 minutes before my flight (so I wasn't sitting around there all that long). What I noticed while I was there:

I was able to observe the full name and address for 3 people as they had luggage tags on their carry on luggage which had their name/address visible to all. This doesn't count the other people who had tags, but they happened to be face down, so I don't know what information was on the tag. My recommendation is to either a) use a tag that covers the information, place the information inside of one of the exterior pockets (I put my business card into the top external pocket) or just don't put anything on carry on luggage as you don't need to.
I was able to observe the name, airline status and account number on several people as I stood in line for the flight. While this isn't as much information as your complete address, I could easily wreak havoc with your travel plans calling the airline to cancel or rearrange flights or otherwise do interesting things with your airline points. What should you do: Remember that this information is on the boarding pass and don't show it off to everybody standing in line next to you. I keep my boarding pass in my shirt pocket printed side facing in or I keep it inside of the carrier until I'm up in front of the line. Note also that this information is printed on the portion of the pass they let you keep. Don't leave them lying about. Trash them like you would trash any other receipt.
Several people had those travel document/ID holders thinking that they are doing what frequent travelers do (which, of course, you never see a frequent traveler use). Problem is that they leak information like crazy. Most people that use them keep their driver's license in the clear holder. So all the way through the security line and while they are sitting around at the airport, anybody who wants to (and has good eyesight) can read all the information there (name, address, dob at least). Putting the passport in there just brags to the world that your a citizen of whatever country (yeah, for some of us that may be obvious, but there's no reason to confirm it for people who don't need to know it). I strongly recommend against using one of these things. If you just have to have such a holder, I would face all the documents in so that you control who gets to see them.

Moral of the story: Be aware of all the places that you leak information and minimize them just as you would want providers to minimize the amount of data they collected about you. Leaking such information opens you to potential stalking, identity theft or other non-fun activities.

Tags : identity / travel / identity theft

Scripts, Browsers and Security

2008-09-06T05:19:00.000-07:00

We all know that many of the common security exploits with browsers is accomplished through the use of the enhanced scripting/programming capabilities such as JavaScript or flash.

These usually aren't attacks on the browser itself, but rather are attacks where the scripting capability of the browser is used to take advantage of an existing session in another window. For example, one attack which was launched via email that included a link to a page which had javascript which opened a hidden window that went to a financial site and tried to make some stock trades. If the user happened to be logged into that institution in a different browser window, the script succeeded in selling/buying some stocks (as part of a pump/dump scheme). Sure, many people did not have that particular financial institution open at the time, but with enough spam, enough people (who should have known better) clicking on links in the email, the fraudster could generate enough successful traffic to enable their scheme.

How does one protect themselves against such attacks?

Turning off such capabilities will render many, if not most, web sites unusable. Turning on and off as necessary will make your browsing unusable for even the most patient user.

If you're running Firefox, there's an add-on you can get called NoScript which makes it pretty easy to manage which sites are allowed to run scripts and which sites are not. I've been using this for a few weeks now and while it was a little tedious at first (each time I went to a new site that used such scripts they would start out blocked and I would have to enable them with a simple click on the notice bar). I could choose to enable all scripts on the page (if I was lazy) or just certain scripts from certain parties that I trusted. I could enable the scripts permanently for sites I visited often, or only enable them temporarily for a site that I was just visiting as the result of some search.

This model makes it much less likely that I'll be surprised by some hidden script on a page that I pull up as the result of a Google search.

A very positive side effect is that those flash adds that I hate so much, are also blocked! Yeah!

I definitely recommend NoScript and what's really cool is that it's free as well.

Tags : Firefox / NoScript / security

Firefox 3.0

2008-06-22T04:27:00.000-07:00

I've been using the new Firefox 3.0 browser for several days now and I have to say that I am very impressed with it.

The browser certainly feels substantially faster at loading the same pages that I frequently visited before with Firefox 2 and with Internet Explorer 7 -- though I have to admit that this is very subjective and that server and ISP performance come into play.

I was a bit concerned that the browser address bar no longer indicates the SSL status of the site I'm visiting. This was a conscious decision by Firefox developers and there are work-arounds for getting it back. I'm not sure I agree with the arguments either way yet, but I was a bit surprised when I first went to a site and the SSL status was no longer indicated in the address bar -- I had to go checking a bit to make sure I was where I thought I was.

The problem that drove this change was the fact that they are now indicating site status with the background around the site icon that shows to the left of the location bar and the colors for the three states (unsecure, ssl and ev) did not match the colors of the address background. The ultimate decision was to leave the address bar uncolored and solely rely on the icon background. While I agree that if there are colors on the background that they must be the same, I think I disagree with not also reflecting the colors on the address background as well. You can read more about this change and the logic/discussion behind it here.

The only problem I've observed so far is that the ordering process at the AT&T wireless site does not work (my son lost his phone and I had to order a replacement for him). I can't explain clearly what the problem is as the site just behaved incorrectly -- asking me for the same information multiple times or doing other strange things like showing nothing in the cart after I selected a phone, clicked on add-to-cart and the popup that resulted from that operation would show that no phone was put into the cart). I tried logging out and logging back in and restarting the browser -- all to no avail. I ended up having to use Internet Explorer to make the order.

Overall, I'm very satisfied with the new version of Firefox and recommend that others using earlier versions of upgrade -- you will like the speed improvements. Congrats Firefox team on a great upgrade!

Tags : firefox / browsers / AT&T / Internet Explorer / ssl

Back-to-myMac brings the Mac Back

2008-05-12T05:34:00.000-07:00

An interesting story coming out of White Plans, NY talks of a woman who's apartment was burglarized with close to $5,000 of electronics stolen including a couple of Apple laptops and how she was able to help catch the culprits as well as get her stuff back.

The thief apparently was using the computer and one of the victim's friends (who knew her laptop was stolen) noticed a few days later that she was logged in (presumably on some instant messenger) and called her.

The woman was able to use Apple's "Back To My Mac" application on another computer to get control of her stolen laptop and activated the camera in the laptop, taking pictures of the thief. A quick review with her friends and they figured out that the guy was a friend of a friend of one of her roommates who had been at the apartment a few weeks before.

A quick call to the police and they arrested the thiefs as well as getting back most of the stolen electronics.

I'm guessing that she's happy she wasn't one of those self conscious users who tape over the camera to keep something like this from happening and I'm not at all worried about the thief's privacy violation. Of course, the chances of other thiefs really being this stupid to make use of a stolen computer without wiping it clean are probably pretty low, so I'm not sure how often this kind of think can happen (but you know they still do give out the annual Darwin awards).

Tags : apple / mac / back to my mac / privacy / darwin

I've been cleared

2008-05-01T07:37:00.000-07:00

I've joined the US's Registered Traveler Program. Clear (a subsidiary of Verified Identity Pass Inc.) operates the local facility here at Dulles, so I joined their network.

In exchange for submitting to (and paying for) a background investigation and biometric authentication (fingerprint in my case, though they also had iris scanners there it wouldn't work on me) you get to have a very short security line -- though you still go through the same "take-off-your-shoes" security process. They seem to be working on getting some form of scanner approved for scanning shoes while they are still on your feet, but recently they did not pass the TSA testing that was done.

The cost of the program from Clear is $128 (at least for the first year -- they weren't clear on what subsequent years will cost). $28 of that goes to the TSA for the background investigation and the rest to Clear (of which, I'm sure, a portion goes to the airport). You can extend that by a year when you use a discount code (and the party who gave you the discount code also gets a year -- so there's something in it for everyone. My discount code, if you're interested in getting a free month added to your subscription is: DSCAM1142273 - use it to your hearts content.

One might ask why I would join the program given that I was already a United 1K member and able to use the premium passenger lines at Dulles. There were several reasons including:

The premium lines are only available when the flight you a premium member on the flight you are leaving Dulles on. I have seen the staff turn away United premium members when they were booked on some other airline where they did not have status.
Even with the premium lines, you can still get stuck in a slow line (I have waited as long as a half hour in the line) and if I'm tight for a flight, that can be too long.
The program is available at other airports (though I'm not sure if I'm only able to use it at Clear supported airports or any registered traveler airport) and in particular, San Jose Airport -- which has no premium lines and where the line for early morning flights can be crazy long -- is one of the Clear supported airports.
I travel often enough that the time savings, even if small, is worth it (in my opinion).

Of course, about 2 weeks after signing up (and paying), I received an email from Marriott (where I am a Platinum member, of course) with the following offer:

So, after I signed up I found out I could have gotten it for free. I called them expecting to get the "Gosh, I'm sorry, but it's too late now" and was pleasantly surprised to hear "No problem sir, we'll just extend you another year". Good deal!

You might be wondering what I think now that I'm a member. I've used it on 5 of my last 8 flights (Portland International Airport does not yet participate in the Registered Traveler program), nor is it available at foreign airports. In Dulles it's down with the employee security line it workes great -- in fact, the people from Clear are almost too helpful (trying to help gather things ready to go through security). I've timed it with another person who was going through the premium security line and I was about 10 minutes faster then them at a time when the lines were short. In San Francisco the clear area opens at the front end of the regular security line, emptying directly into the xray scanners (so, essentially, you jump to the front of the line).

One thing on the negative side, if you're traveling with people, you can't bring them with you, so either you have to go through the regular lines, or you have to split up. That already happened to me when I was traveling with George on our way to the European Identity Conference in Munich -- that's how I figured out that there was a 10 minute difference (it was an experiment!). Not sure what I'm going to do when I am traveling with my family to England & Ireland this summer -- I don't think my wife will be as easy going about it as George was :-).

All-in-all, I'm a happy customer..... And remember, if you want to sign up, use the discount code "DSCAM1142273" so we both can get a free month :-).

Tags : identity / travel / dulles

Updated Liberty Open Source

2008-02-18T15:24:00.000-08:00

I've updated my Liberty ID-WSF Open Source Toolkits again. This time to reflect the minor changes made in the Advanced Client specifications as they were finalized within the Alliance.

For those of you who aren't familiar with this code, I have two toolkits available -- a C++ client and an Axis1/Java Server -- which implement the Liberty ID-WSF protocols (both the basic framework and substantial portions of several services).

This new release of the toolkit does not add new functionality -- it only brings the code up to match the final specifications.

Have fun!

Tags : identity / liberty / Liberty Alliance / Open Source / ID-WSF / Provisioning / Advanced Client

What's wrong with this picture?

2008-02-16T17:21:00.001-08:00

I went to login to my discover card account to review my account activity (something I try to do on a regular basis). Using a bookmark (to make sure I don't accidentally enter a typo that gets me to a hackers site -- plus I'm lazy and a single click is easier than typing in the URL), I get to the web site and I notice something that isn't right (in my opinion). Take a look at the picture below and tell me if you see it before reading past it.

Look at the URL. It's non-SSL (http: vs https:). When I noticed that, I figured that somehow my bookmark was messed up, but looking at the bookmark, it does specify https:. What happens is that Discover is redirecting you from the SSL endpoint to the non-SSL endpoint. This happens with IE and with Mozilla whether directly connected or through a proxy server, so it's clearly something done on the server and not a side effect of the client.

That wouldn't be all that bad if Discover just had a link on the home page directing me to a login page that was SSL protected. That isn't the case. The home page prompts for the user's credentials. Now the technical people out there might say that the data from the login form is probably submitted via an SSL endpoint so the data is protected. However, without looking at the source code, the user can't know that.

In addition, since the URL itself isn't protected, the user (me in this case) doesn't have any way to know that they are actually talking to Discover. This could be a MITM phishing site.

So, if you do go to Discover's site to view your account, I suggest that you select the login link in the upper right corner before you enter your credentials. This will bring you to an SSL protected page where you can verify that the host you are talking to is discovercard.com and not some MITM.

Tags : identity / id theft / ssl

Time Machine

2007-11-19T04:12:00.000-08:00

One of the current "joke" emails flooding the internet is an email showing pictures from a 1977 JC Penny catalog. Given that the email referred to "blog fodder" I decided to search around and I've found the original post. Definitely worthy of a read.

Strap In, Shut up and hold on -- we're going back.

If you remember this stuff... If you wore this stuff... I'd suggest that you not share those identity attributes -- unless you don't mind being the butt end of many jokes for the rest of your life.

Tags : identity / time machine

Anti-gulllibility training

2007-11-15T05:18:00.000-08:00

I've always felt that one of the most important tasks for a parent is to teach their kids to not be gullible. I routinely work on such training with my kids. In fact the other day, I was way into the story about how Los Angeles schools, while not getting many snow days, do get closed for bad hair days. Unfortunately, while my daughter was well into the "realy?" stage, my wife piped up with "They do not!" cutting me off at the knees.

Nothing is a better example of the importance of such training than the comment in response to Paul's revealing post about Microsoft's Identity Assistants.

So parents, take this as a warning. Train your kids in anti-gullibility before they make a fool of themselves publicly.

Tags : identity / Microsoft / Microsoft

Madsen's Lemmas (or is it Lemmi)

2007-11-07T10:38:00.000-08:00

Paul writes about attributes and how they won't be trusted for self assertion when the value of the attributes is used to distinguish levels of service.

In the context of any given application, a Relying Party will be unwilling to accept a self-asserted identity attribute without verification if there exists the possibility of differentiated advantage to the user in claiming one value for that attribute over another.

And follows with the corollary:

For any given identity attribute, there exists an application context in which there can be differentiated advantage to the user in claiming one value for that attribute over another.

Combining the two would make one think that Paul is arguing that self asserted identity attributes will never be accepted, but I'm pretty sure he didn't mean that.

In any case, I think there's another side to this puzzle in that the self asserted attributes can be accepted and used when the result makes it useless for the user to lie about them. If I order something with Paul's credit card, name, address and phone number, it generally will be accepted, the transaction will complete, and the vendor will ship the product -- it will just end up at Paul's house rather than mine, so I won't benefit from it (but I bet Paul was surprised when those enlargement pills showed up :-)).

So I would write the lemma more along the lines of:

There exist some set of cases where a Relying Party provides such differentiated levels of service that they will require third party attestation and/or confirmation of attributes in order to enable access to such differentiated levels of service.

PS. Paul, if you need to fake your IP address to make it look like you're coming from the US, let me know... I can give you access to my proxy server (without, of course, any guarantees as to snooping on the traffic :-)).

Tags : identity / attributes / self-asserted

Living without flash....

2007-11-02T05:50:00.000-07:00

Back in March, I wrote about finally succumbing to the need for add blocking when flash adds on several sites were measurably impacting the performance of my system. When I reloaded my system I decided to forgo installing the flash player as my solution as the add blocking software was still kind of a pain.

Well, after a month or so of living without a flash plugin it seems I have to reverse my decision. Too many sites out there are totally unusable without flash. Many use it as an integral component in their site navigation (try researching Dish Network's offerings or look at SciFi's channel info (2/3rds of the home page is blank with "this section requires flash")... Others use it for processing particular functions (I can't us discover's secure credit card number generator because it only works with flash, I couldn't order my daughter's school yearbook from Jostens because the required personalization step requires flash with no alternative).

While all this glitz is nice for the marketing guys, I think that this is a bad thing. Especially when you consider that flash doesn't work all that well for accessibility (just imagine the blind person trying to make sense of the glitzy flash driven site navigation system). The Web Accessibility in Mind folks have a good article on accessibility programming with flash but they note that it's hard to do well.

My suggestions:

Never use flash for site nagivation. Javascript works well enough.
If you do use flash, provide reasonable alternative, keyboard based, means to obtain information from your site.
Evaluate the accessibility of the information and make use of the suggestions provided by WebAIM.

Ideally what I would like to see is an option in Firefox to manually enable flash processing on a site by site basis -- those sites that abuse the privilege by writing CPU intensive flash apps would be blocked, while the more typical mundane implementations could be allowed.

BTW - Given that no browser includes flash out-of-the-box (it's always an add-in plugin as far as I'm aware) I now have some good ammunition to use when I run up against those that resist authentication models requiring software on the client.

Tags : flash / Macromedia / Adobe / WebAIM / Dish Network / Discover / Jostens / SciFi / Firefox

New Gadget #13

2007-10-22T13:40:00.000-07:00

My latest gadget is an update on a previously reported gadget.

This week I bought the latest and greatest Western Digital Passport external hard drive. A drive with a honking 250GB of space on it in the same packaging that my older 120GB and later 160GB drive used. In fact side by side with my prior 160GB drive you can't tell which is which:

Interestingly, they came out with this quick enough that they are still using the 160GB retail packaging with just a sticker over the 16GB on the front of the package. When I first looked at the back of the package (which listed only 120GB and 160GB) I was worried about a bait/switch from the retailer. However, that wasn't the case, it was a real deal.

The drive comes formatted with a FAT filesystem and has software for doing automated backups and synchronization with your primary hard drive. I immediately reformatted it for NTFS so I could use encryption and such on the drive. I don't need the backup or synchronization stuff as I use this drive as an extended primary drive rather than a backup drive. I use rsync to backup my system (including the WD drive) to my server regardless of my location (remote or at home).

Some who have used this device have complained about the fact that it sometimes won't work in their USB port. WD does document that it requires a full power USB port (though I can't find any documentation on exactly what is a full power USB port and how do you know you have one). I have had problems when I plug this device into some ports and found that on my laptop only one of the ports works reliably. Even the ports on my external powered hub are not sufficient to power the device alone. So when at home using the hub, I use a Y cable that grabs power from a second USB port to power the WD drive. I'm not sure where I got the cable, it was lying about in my USB cables collection, but WD does sell one.

The only other thing is that I suggest you buy the slipcase sold by WD to protect the drive when on the road. I had one lying about from my 160GB drive, so I just used that one.

Tags : disk / Western Digital / passport / external hard drive / gadget

Subversion end-of-line style

2007-10-21T17:59:00.000-07:00

In my work with the Liberty Alliance, I'm the editor for several documents in the upcoming Advanced Client specification set. We use subversion as our source code revision control system.

Recently, when I was working on a new draft of the specs and committing a set of files that included a number of Visio drawings and the equivalent Encapsulated Postscript file images, I ran into problems. After all the files were uploaded during the commit, which failed with the error message:

svn: File "xxx.vsd" has inconsistent newlines svn: Inconsistent line ending style

A quick look at the file and I saw that the files had the typical windows line terminator CRLF rather than the UNIX typical LF. So I hand edited the file removing the CRs and tried the commit again. The same thing happened just with the next file in the list. So clearly this was going to go on for each file. So I did what any other UNIX weenie would do -- entered a one line shell script for loop on the command line using tr to delete the CRs in each file.

This got me past the problem and the commit succeeded. However, I was not totally satisfied as I wasn't sure that if they could be edited in Visio with these changes. So I dug a bit deeper into the problem looking into Subversion.

It turned out that subversion has attributes on files, one of which is "svn:eol-style". In this case, the files I was working with had gotten this attribute set to "native" which on the UNIX system I was on would be "LF". Not good for a file from Visio. I thought about changing the svn:eol-style to CRLF which would get around my specific problem at this time (until Visio changed their file format), but the better solution ended up being to just delete this attribute on the file with the following command:

svn propdel svn:eol-style *.vsd *.eps

Then I copied in the files from my Windows partition (where they still had the CRLFs) and committed the files without a problem.

Tags : subversion / visio

Checking in too early

2007-10-16T23:54:00.000-07:00

Like most airlines, United has, and strongly encourages the use of, an online check-in tool so that passengers can check-in for their flight before leaving home. This is seen as a win-win situation for everyone. United gets the user to do the manual labor of checking in and paying for the paper stock for printing the boarding pass while the user gets to avoid check-in lines at the airport.

I am a big fan of using this and typically check in near the limit of 24 hours before departure. I check in this early in part so that I don't forget to check-in in the mad rush out of the house on the day of my flight and in part so I can check to see if there's a better seat available at check-in.

However, this has led to one problem. On several recent flights, I was upgraded sometime between my early check-in and my departure for the airport. Because I was already checked-in in coach, I was unable to select a seat in the first class section. Theoretically I should be able to un-check-in and then re-check-in, or I should be able to get the customer service people to do the same for me, but neither worked and I had to wait till I got to the airport and the gate agent opened the flight at the gate (even the Red Carpet Club agents were unable to help me).

Moral of the story: If you're on the upgrade list, don't check in till you're close to leaving for the airport.

Update (10/21/07): Not learning from my own mistakes, I checked in around 11:30 PM the night before a flight to Tokyo as it appeared that there was no chance that it would clear before leaving for the airport in the morning (it was a 12:41PM departure). Of course, I was wrong and the upgrade cleared at 5:03 AM. But, since I was already checked in, I couldn't select seats in business class. Checking the site (by the usual trick of trying to purchase a business class ticket) showed that there were still 4 seats open including a coveted aisle seat (8D). By the time I got to the airport and checked in, the only seat left was 13E (a middle seat). Hopefully you'll learn from my mistakes better than I do.

Tags : United / UAL / Easy Checkin / travel

A broken Washer

2007-10-14T06:52:00.000-07:00

This is a long story, feel free to just cut to the end.

A little over a year ago, we bought one of Sears top-of-the-line washing machines (the Kenmore Elite Oasis Canyon) for several reasons including that it was EnergyStar compliant while also being very large (so we could do our laundry in less loads while saving energy).

We really like the washer. It does a great job on our clothes, does it quickly, and does lots and lots of clothes at the same time, while also being very efficient at doing small loads.

However, we didn't like the fact that the thing just up and died mid-load with no sign of life in it. None of the lights were lit, none of the buttons did anything. On top of that, the lid was locked and there was nothing I could do to unlock it, so our clothes were stuck in there. Power cycling it did nothing (though I was able to use a meter to verify that it was not only getting power, but also consuming some small amount of wattage). Just in case you're wondering, no, the there was no surge on the line as the weather was clear (we're in the middle of a drought) and I have UPSs all over the house which beep like crazy for any power line problems -- none of that happened, so I'm pretty convinced it was not a surge.

Of course, the warranty was over (1 year) and, given that it was a top-of-the-line system I thought it wouldn't be necessary to buy the extended warranty (especially since they almost always are a waste of money). So the repair was going to be on us.

We called Sears Home Repair and they couldn't schedule someone to come out and fix the washer for 2 weeks. When he did get here, he determined that the electronics module behind the console was bad, ordered a replacement and scheduled someone to come out and install it (another 2 weeks later). This at a cost of $346.77.

The part came in a few days and since I didn't want to wait another week to get a working washer (we had already been to the public laundromat once) I tried to install it (something not all that unexpected if you read my blog). However, the cover over the board was screwed down with 3 screws and had 2 locking tabs. I was unable to get the locking tabs to release no matter what I did. I gave up and decided to wait for the repair guy to show up.

This past Thursday he shows and he had the same problem with the tabs and ended up cutting them off. After installing the board, the washer was still dead. He then said that the problem was most likely the main electronics unit (motherboard to the rest of us) and ordered one with "Emergency" delivery and scheduled a return visit the following week. He also noted that the replacement board was a different part and therefore they probably had fixed something in there. This at an additional cost of just under $300, bringing the total to $625.25. Needless to say I was NOT happy.

I poked around on the web and on Sears own site, found several people who had complained about this same exact failure just after the warranty expired. It seems like this was more of a general problem than a unique failure.

Armed with all of this information, I called customer relations and after about 40 minutes on the phone I was asked if they could call me back. I was hesitant because I was afraid of not getting a call and having to start over, but I went along with them. About an hour later, she called back and said that she had found that the electronics were covered by a 2 year warranty and the repair would cost us nothing (and she arranged for a refund of the initial charge on the first visit).

The part came in on Friday and since I didn't want to wait another week for a working washer, I decided to install it myself. This was a bit more complicated than the first board as it had many wires running about, but I took a few pictures so I could verify where all the wires should be and off I went. After about 15 minutes, the module was in, the washer was all back together and magic, I had a working washing machine. Of course, I was again proud of myself for doing the repair (though it was much easier than when I replaced the LCD on my camera).

One thing I did note once I had the washer working again: when I do a load a bleach load of whites on hot, steam comes out around the lid of the washer. Some of this steam could leak into the area with the electronics if the seals aren't tight enough. It was a similar load/settings on the washer when it died.

Things to learn from this

The squeaky wheel definitely gets the oil. It was only after calling and talking to 3 people at customer relations did I get to someone who magically said "oh, that should be covered by warranty". I do have to admit that I can't find any such warranty statement with the documents for my washer, but I'm happy to get the part fixed.
I was amazed about how little trouble-shooting was done on the phone prior to rolling a truck. Dell is a pretty good example for how to do this right, they will work quite well over the phone to figure out the exact problem to save a truck roll if possible. A little diagnosis over the phone and they would have known that the problem was with the electronics and could have sent the parts so that the unit would have been fixed the first time (note that Dell will even let you install the parts if you feel comfortable doing so -- which I've done several times for keyboards and such stuff).
Sears definitely has a problem with the electronics module for this unit. When something like that happens to a car, they do a recall, or they do a proactive warranty extension to keep their customers happy. Sears doesn't appear to be doing this and that's problematic given that they risk loosing a customer who is buying their top-of-the-line (probably widest margin) goods.
The lid-lock should release when power is removed from the machine. Having it stay locked like that meant that we were unable to remove our clothes from it until the repair guy came and took the machine apart (and the lid was still locked when he was done, but we did get our clothes out).

Update (10/21/07) - apparently things were not as well worked out as they appeared. The service guy still tried to come out to my house for the installation of the part even after I had twice called to tell them that I had installed the part and they had said something to the effect of "Cool, then no need for us to come out. I'll cancel the service call". The service guy said he still had to come out to collect payment. I told him that customer relations had said that this was a warranty repair as the electronics were warrantied for 2 years. He went off the check on this and called me back about a half our later saying that that was a parts-only warranty and that since I had installed it myself that voided the warranty and that I would have to pay both for the service call and the parts. I told them to give it a shot, but that there was no way in hell I was going to pay for this work. We'll see how this works out.

What I can't understand is how Sears is showing that they have no interest whatsoever in smoothing things over with a customer who has routinely purchased their top-of-the-line appliances. Ruining a relatitonship like that over $65 or so just doesn't make sense to me, but that's what they appear to want to do.

Tags : sears / repair / EnergyStar / appliance / Dell

The Case for Federation and SSO

2007-10-06T12:07:00.000-07:00

To date, the vast majority of real-world federation roll-outs have been internal or enterprise type deployments. Things like an enterprise authenticating its users out to an outsourced provider (such as a Fidelity 401K, or AOL's Radio Service). Yes there are many exceptions to this general statement (you can see many of them on Liberty's Adoption Page), but that is the general view of the industry and I certainly don't knowingly use federation in any cross-provider operations.

The time has come for federation and Single-Sign-On to be adopted in a more general fashion. I say this for many reasons and hope that the various vendors and providers out there will not be stubborn and/or resistant about it. I think this is valuable to parties that will wish to assert identity. I think this is valuable to the people who will accept identity federations and I think this is valuable to the users themselves.

When I say it is valuable to the user, I don't mean the often quoted "that way you can reduce the number of passwords you need to remember" -- though I still think that is a reasonable benefit. The real value for the user is that they will be able to share their data across multiple providers without the need to give their credentials to the other party.

Examples that already exist today include:

On LinkedIn, if I want them to pull my contact information from my contact book in several mail services (e.g. GoogleMail, HotMa il, etc.) I have to provide LinkedIn with my username and password on the mail service. LinkedIn logs in as me (either through their web interface and does screen scraping, or directly via an exposed web service) and extracts my contacts. Since I gave them my login information, I'm hoping that they don't do anything wrong with the data (like expose it), and that they don't mis-use the access to my account (e.g. sending spam in my name).
On Etrade, when I want to setup a new bank account for transferring funds from my Etrade account, one of the options provided is for ETrade to be provided with my username and password for online access to my bank account so that they can verify that I have control of the account and that it is in my name. Like LinkedIn, I'm hoping that they don't do anything wrong with the credentials while they have them (and in the case of ETrade, hoping that they do not store them like they claim they won't do).

I could go on with this list, but you get the idea. The user is already federating their data together across different providers. It's just in a very broken way that can lead to cascading security failures as any security failure at one site can lead to security failures at other sites.

With federation, I wouldn't need to give my credentials to LinkedIn. My mail provider could also differentiate the access provided (letting LinkedIn see the set of contacts that I chose to share with LinkedIn without being able to send mail in my name or being able to change contacts). LinkedIn could maintain that federation so that they could periodically check for updates. A break-in at LinkedIn would mean that someone could perform the same operations that I've already OK'd for LinkedIn -- get the data that LinkedIn already has -- so no additional exposure.

Why would GMail, HotMail, or even my bank, want to do this? First off, they are already doing it in an insecure way (I can always give my login credentials to the other party) and with the expanded access at their service. This would be a much better solution from a security and least priviledge point of view.

Another issue that might be raised with regards to the service providers is why would they want to expose a web service with this data. In many cases that's a new thing for them to do. But I think it's worth it becuase today, when they don't expose such an interface, theh other parties just walk though their standard user web interface and do screen scraping of the data -- I'm sure that data via a web page is more costly than exposing it through a programmatic web service.

Of course, LinkedIn would want to do this as they already do, but within the restricted capabilities of today that open them to some liability as well (should my data be misued at their site).

While I spoke heavily about LinkedIn in this post, this clearly applies to any and all cases where I want to do things across sites -- this is becomming more and more important in the Web2.0 world more interesting applications join togetether information from various parties. I can see how Dopplr would want to access my LinkedIn account to get my list of friends to pre-populate my traveling buddies rather than me having to establish new connections. I can also see how Dopplr would want to get access to my United Airlines itineraries so that they could auto-populate my trips.

The list goes on and on and it's a win-win-win for everyone, users and providers.

You might then have the decision as to what token format one should use for the federation and what web services structure one should use for the service access. I, of course, would recommend SAML and Liberty's Identity based Web Services Framework (ID-WSF), respectively, but that isn't as much the issue as is getting this up and running for the users.

You might notice that in this case, I haven't been advocating a large Circle of trust with centralized IdPs. Most of the examples I gave were point to point federations where, essentially, the relying parties and the IdPs were the same entity. The advantage with this model is that you have no need for extended business agreements so it's much easier to roll out. I do think that as more and more people start adopting and using this, it will be a natural evolution to environments where there are separated IdPs and Relying Parties, but we don't need to start there.

Tags : identity / federation / linked-in / ETrade / UAL / United Airlines / SAML / Liberty / Liberty Alliance / ID-WSF

A painful Vista

2007-10-03T14:14:00.000-07:00

My Dell Latitude D830 came with Microsoft Vista which, for the most part, has seemed like a prettied up XP without a lot of added useful functionality nor a substantial increase in stability. At the time I upgraded, I wrote that I was like so close to buying a Macbook Pro. I am sorry to say that I regret that decision even more today.

What got me to that stage? Well, it's a long painful path and to be honest, I'm not at the stage (yet) that I'm ready to just give in and replace my fairly new laptop.

The problems all started about 2 months after receiving the laptop. On July 14th, the day before I left for a trip to Shanghai, my email program (Thunderbird) locked up. When I restarted it, it still had problems and wouldn't pull mail from nfthe server, so I shutdown and restarted the computer.

During the reboot, the OS decided a chkdsk of the NTFS filesystem was necessary and it found and fixed many problems. When I got back to the running OS, all of the files that were actively open at the time I cleanly shutdown the system were gone. Totally gone. Not in the found.* directories (the NTFS equivalent of the UNIX lost+found directory).

Luckily I had the data backed up earlier that day as well as an offline backup on an external drive from a week before that. Since I was taking off for Shanghai, I copied both backups onto my system so I could pick the files (as I wasn't sure if the backup earlier that day wasn't corrupt as well).

I was able to get up and running again on my way to Shanghai without a problem and things were working fine. I assumed it was just some freak accident.

About a month later (mid-August) the same thing happened. This time I dug into it further and found that there had been a series of events in my event log (both then and back in July). It seems the problem starts with an NTFS event (which is flagged as an "Error" rather than a "Critical" event) with the event code of 137. The message from the event was extremely helpful... NOT!:

The default transaction resource manager on volume D: encountered a non-retryable error and could not start. The data contains the error code.

Microsoft's online help for the event was no help:

Results for: Microsoft product: Windows Operating System; Version: 6.0.6000.16386; ID: 137; Event Source: Ntfs;

No results were found for your query. Please see Search Help for suggestions.

Googling on "Default transaction resouce manager" found little results as well, but there was at least a possible link to another's problem. Apparently some had discovered that Acronis True Image had led to similar problems. I had installed Acronis Disk Director to reorganize my disk partitions, so I uninstalled it to see if that would alleviate the problem. And, of course, I did the same restoration process to get back all the lost files.

I did find one interesting discussion on resource managers in Vista, but that didn't provide any information that would help solve my problem.

Given that the error message just showed up in the event log (and in both cases, was close to 24 hours before the system crashed -- allowing me to open/use many files that disappeared), I added an event alert task which would send a message to the console should this error occur again. This is really important so that you can catch the problem as it starts, minimizing the potential damages.

Things went well for about another month and then it happened again in Mid-September, so it clearly wasn't the Acronis product. I was busy getting some heavy work done, so i didn't have the time to explore the problem other than to restore the files again.

About a week later, it happened again. This time it started going into an almost daily problem, sometimes happening again just after I had fixed things and ran chkdsk to fix the problems.

The pain had passed the threshold and I decided to do a total reinstall of the system. Prior to doing that, I did a complete backup. I copied my data files to a portable drive. I ran the extensive system diagnostics including the full suite of hard disk diagnostics to see if there was some form of a hardware problem. All diagnostics passed.

So, this past weekend, I reinstalled vista. I've been installing each of my former tools (there are many of them) and so far, so good. Given that this didn't turn up until I had had the computer for about 60 days the first time, I guess I won't know for sure if I've gotten around the problem till early Dec.

And, of course, I went and added the event task to generate a message should this occur again.

Wish me luck!

Tags : vista / Microsoft / NTFS / Dell / Latitude / Acronis / Mackbook Pro

Mistaken Identity

2007-09-10T11:55:00.000-07:00

In a case of mistaken identity (of a place rather than a person) my United flight to Portland last night was delayed and had to re-connect the jetway and switch passengers.

As we were getting ready to push back, one of the passengers got up and talked to the flight attendant in the front of the plane. They talked, she talked to the pilot, they talked some more. All I could here was "I'm really sorry" coming from the passenger.

The jetway re-connected and the guy got off the plane (while another passenger, who had been denied boarding because the plane was oversold, got on -- lucky him).

Apparently, the departing passenger had booked the tickets, checked in, and boarded the plan without realizing that he hat ticketed himself to go to Portland, OR, rather than Portland ME. I would have thought that the 5 hour flight time would have given him a hint, but perhaps the fact that the 3 hour time difference made the apparent time difference (if you didn't pay attention to timezones) appear to be just 2 hours.

Anyway, luckily we hadn't gone far and he didn't have any checked baggage (nor, from what I could see, much carry-on luggage -- just a small laptop case). So after the quick switch we were on our way.

Tags : travel / ual

Advanced Client Take 2

2007-09-05T07:47:00.000-07:00

The second draft of the Liberty Advanced Client Technologies set of specifications has been published on the Liberty Alliance web site.

For those who aren't aware, the Advanced Client Technologies work is the 3rd generation of client technologies coming out of Liberty. The first generation was work that enabled a Liberty-aware client and/or proxy to participate in the SSO transactions (similar to what Cardspace does today). The second generation enabled active clients to act as WSC's in identity transactions (such as a radio or mail client authenticating with an IdP, discovering and accessing a service provider).

This third generation enables clients acting as an extension of network providers such as an IdP, and addresses the issues related to hosting full-fledged service providers (such as my own IdP, or my own Contact Book Service) on my personal client.

So, this is your chance to nail me to the wall and point out how many stupid things I've done in there (though I'm not the only contributor, I'm sure that if something stupid is in there it is my doing). Please take a look-see and let us know of any interesting things you find in there (even pointing out the many, I'm sure, English mistakes would be helpful).

Go for it!

Tags : identity / Liberty / Liberty Alliance / Advanced Client

Portals and IdP Discovery

2007-09-04T08:07:00.000-07:00

I recently received a comment on my SAML Bashing blog entry. "Jeremy" (not sure which Jeremy as he was otherwise anonymous in his comment -- I wonder if it's really James in disguise -- this seems the kind of comment James would leave, but James is usually quite blatant about it, not hiding behind an identity pseudonym) asked:

Kim stated "The question of how the relying party knows which identity provider URL to use is open ended. In a portal scenario, the address might be hard wired, pointing to the portal’s identity provider. ". What are your thoughts on that?

In the early days of Liberty ID-FF, we paid a good amount of attention to what solutions would fit into the various portal solutions. Must of this has to do with the configuration and structure of the portal. We saw different portals using different solutions including:

Push SSO

In "push SSO" the portal, when creating links to the various components that make up the portal, generate redirection links that send the user directly to the IdP with some additional information causing the IdP to initiate an SSO to the third party.

This is a common solution used in enterprise portals when the user selects a link provided by an outsourced third party (such as Fidelity providing 401K or stock purchase account management for employees).
Well-known IdP

This is the solution mentioned in your quote of Kim. The members of the portal know which entity provides IdP services for the portal and can send the user to the IdP to get them authenticated. This is how most portals work today (e.g. Yahoo's IdP is known as the IdP for all Yahoo services at the Yahoo portal, so when I go to Yahoo Games, I get authenticated by the Yahoo IdP).
Affiliations

Affiliations are a technical structure used to represent provider membership in a group (such as a portal, but can also be other business groups). When the user "federates" to an affiliation, the members of the affiliation are able to treat the user a a common user providing synchronized services and precluding a multitude of consents and idp interactions.

The concept of Affiliations was introduced in the ID-FF specifications and was incorporated into SAML 2.0 during the convergence of SAML 1, ID-FF and Shibboleth.

Tags : identity / IdP Discovery / Liberty / Liberty Alliance / affiliations / SAML / Yahoo / Fidelity

Relatinships and authorization

2007-08-24T06:16:00.000-07:00

James McGovern writes about how relationships must include authorization:

Anyway, the notion of relationship is something that belongs to the identity provider and entities such as the Liberty Alliance are defining standards around it. Check out their notion of the people service. The key though is that relationships sometimes require authorization. For example, just because my son can order an insurance card from Amica doesn't mean he is also allowed to cancel the policy for the entire family. Relationship needs authorization especially in domains having to do with medical interactions.

While I like his good words about the Liberty Alliance, I take exception with some of his conclusions.

First off, I don't think that relationships should or must belong to the Identity Provider. This is especially important in a world where my relationships cross the boundaries to many different Identity Providers. Within Liberty's People Service, we took great pains to ensure that the protocols support both a) the People Service be able to be provided by a party other than an IdP (just as LinkedIn provides this type of service to their customers) and b) the relationships contained within a user's People Service must be able to cross identity domains while still protecting the privacy of the users. The latter requirement lead to some rather complex protocol sequence requirements when establishing a connection.

Secondly, I look at authorization as being associated with the object being accessed (where the input parameters may include individuals and/or group memberships) and not with the relationship itself. So in the example provided by James, James would introduce his son to Amica (using the People Service) and then set the associated rights at Amica, not within the People Service. The primary driver for this is that only Amica understands the objects available to Jim and the associated access permissions that may be possible for those objects.

The one place where I see the People Service (and/or any other relationship tracking service) getting involved in authorization is where the user controls what another may do with his relationship (e.g. I can allow Paul to see my relationships (and the fact that I long ago had a coolness link to the ever-cool Joni)).

Tags : identity / authorization / Liberty / Liberty Alliance / Linkedin / People Service / Relationships

Sniffing Cookies

2007-08-03T17:28:00.000-07:00

In Tools to sniff and clone cookies Stephan Brands writes about a scene at a recent Black Hat Security conference where a presenter was able to steal live sessions by sniffing cookies on open internet connections and concludes:

The message for those working on digital identity solutions, in particular “lightweight” identity solutions and plain-vanilla browser identity federation a la ID-FF, should be clear: unless asymmetric cryptographic protection is made an integral part of a solution, users are highly vulnerable to theft of IdP login credentials as well as of identity claims that are issued to them.

First off, to be very clear, there was absolutely *NO* stealing of login credentials. What was actually stolen in that particular case was a session cookie that would enable the hacker to use an existing session for the length of the session. The stolen cookie could not be used to establish new login sessions (as login credentials would allow).

Secondly, in a Liberty ID-FF and/or SAML scenario the authentication protocols are required to take place within an SSL session and we strongly encourage that SSL be used to protect the authenticated session afterwards.

The real example that was shown is that services that do not use SSL to protect communications from the browser to the server are liable to be monitored, recorded, and even hijacked -- regardless of how well the user was authenticated.

Moral of the story: Use SSL to protect communications of sensitive information.

Tags : identity / cookie / Liberty / ID-FF / SAML / SSL

United announces new Business Class Seats

2007-07-23T00:35:00.000-07:00

Today, United Airlines announced their new lay-flat business class seating that will start rolling out into the fleet later this year with a completion of the roll out in 2009.

The new business class seat is a substantial upgrade over the current seats including:

Lay-flat 6'4" bed
15.4 inch LCD display
110 volt outlet -- no more need for empower adapter!!!!
Apple iPod dock
USB power supply to power/recharge devices
etc., etc.

You can take a look at the following for more information:

All I can say is "bring it on!!!" I'm ready for it today.

UPDATE: 7/24 - The down side in all this is that there are substantially less business class seats in each of the aircraft: 747 - 53 (down from 72), 767 - 26 (down from 32), and 777 - 40 (down from 45/49). So while the seats are much better, there are less of them making upgrades much more competitive. TANNSTAAFL.

Tags : United Airlines / UAL / Business Class / travel / iPod / EMPower

Shanghai Maglev Train

2007-07-22T15:12:00.000-07:00

When we arrived in Shanghai, being the techno-dweebs that we are, we just had to ride the MagLev train from Shanghai Pudong Airport to the Shanghai Metro's Longyan Road station.

The train ride is quite short (less than 8 minutes) but well worth the 50 yuan (about $6) just for the experience of riding at 431 KPH at ground level.

I took the video below (about 1 minute long) to show the speed that you see as you're riding along at that speed. This video is not retouched or sped up, that's the normal speed.

Enjoy!

Tags : Shanghai / MagLev / Manetic Levitiation / train / PVG

8 hours in SFO.... and then some...

2007-07-21T17:20:00.000-07:00

On my way to Shanghai this week, I had a layover in SFO that was originally supposed to be around 4 hours. Normally I would fly through Chicago on my way to Shanghai, but my boss and a co-worker were traveling as well and they wanted me to meet up with them in San Francisco. 4 hours is a bit longer than the typical layover for me and part of it was my fault -- I had booked an early morning flight from Dulles because it was internationally configured and so the upgrade to business class was so much better.

When I got to SFO I received a text message from United that the flight to Shanghai was delayed 1/2 hour (to 2:20 rather than 1:50) -- no big deal.

However, that wasn't the end. Around 2:00, they told us that the delay was now changed to 6:40PM (another 4 hours) and they provided us with a meal voucher.

Later (around 5:30PM), they delayed the flight to the next day at 9:00 AM. This wasn't a cancellation, but a delay -- and given how full flights are nowadays, I'm glad they did the delay or it might have taken us days to get to Shanghai. They gave us dinner and breakfast meal vouchers as well as putting us up at the local Hyatt.

So, almost 24 hours after arriving in SFO, we took off for Shanghai and the rest of the flight was uneventful. Luckily, I was upgraded into business class the entire way.

Of course, the Chicago to Shanghai flight that I normally would have flown went off without a hitch and I would have arrived in Shanghai almost 24 hours earlier. For some reason, I find it necessary to remind my boss of that every few minutes :-).

Tags : travel / United Airlines / ual / Dulles / iad / sfo / hyatt / ord / Shanghai

Maintaining Social Networks

2007-07-11T07:35:00.000-07:00

A recent article on the Teknision blog complains about the pain it is to build and maintain social networks again and again on one site or another:

There is something very wrong with the web……

I wonder how many times I have had to find and add Gabor Vida, Steve Mackenzie, Ryan Stewart, Mike Chambers, Phillip Kerman, Mike Downey, Mike Potter, Stacey Mulcahy, Ryan Murphy, Mykel Ruvola( and on and on and on and on) in the last few months. I have spent a huge amount of my time across social networks re-finding the same people over and over and over again.

I too have felt that pain and I am feeling the pain yet again as several of my compatriots have joined dopplr to keep track of where we all are and find interesting crossings of paths as we gallivant around the world.

Interestingly, this is what the Liberty Alliance's People Service was designed to solve, including the connection to people in different identity circles (e.g. they didn't all have accounts within the same identity domain). You can follow along on a webcast on the subject: audio is here, presentation deck used is here.

Take a look, there's some interesting stuff there.

Tags : identity / liberty / Liberty Alliance / People Service / social networks / Dopplr

Harry Potter Mania

2007-07-11T05:44:00.000-07:00

Today, my daughters (Lauren and Jessica) were quite grumpy when heading off to their eventing horse camp (even more grumpy than they usually are in the morning). Their problem stems from a lack of sleep as they were up till quite late last night so that they could see the midnight showing of the new Harry Potter movie "Harry Potter and the Order of the Phoenix".

The movie was great (perhaps even more so since I knew I was seeing it about 3 hours before Eve :-) -- she's on the west coast, though I wouldn't be surprised if she few to the UK just to be able to watch it last week during its world premier). Warner Brothers have done another good job transferring the magic of the book to the screen.

The movie was true to the book, fun and quite enjoyable. It felt a bit long in a few places, but the books are getting quite long as well. Of course, I haven't read the books recently so now I'm all confused about what happens in book 4 vs 5 vs 6. I guess I'll have to go back and read them again before I start in on book 7. We had to order two copies of the last few books in the series (including 7) in our house to ensure a reasonable wait time for each of the readers -- we will all read it.

What amazed me about this movie is that we got there around 10:15 or so (almost 2 hours before the star -- much earlier than we had gotten to Star Wars Episode III's midnight showing and got great seats for that movie) only to find that the theatre was already 2/3 full. The entire middle was pretty much full and we were relegated to one of the sides. Pity the people who only showed up about an hour before start time as they had trouble finding 2 seats together anywhere in the theatre.

Tags : entertainment / Hogwarts / Harry Potter / Warner Brothers / Star Wars

VMWare Tips & Tricks

2007-07-09T05:30:00.000-07:00

I've been using VMWare Workstation for several years now (after dumping VirtualPC when Microsoft bought them and promptly dropped support for Linux guest OS integration). As part of my recent upgrade to Windows Vista, I upgraded to VMWare Workstation 6.0 and had to re-enable all of my tricks to get everything working the way I like within the OS (and I had to find them all again as I hadn't written them down as I discovered them previously).

So, this time, I've decided to document them here so that others could benefit from them (and so I had them lying about for the next time I have to do the same). They are listed here in order of discovery (as opposed to any semblance of an order of importance). I will continue to come back to this and add new things from time to time as I run across them. If there's something of interest you thing should be added, let me know.

My configuration is that I have a Windows Vista host OS and two guest VMs, one running Windows XP Pro (as that is necessary for correct operation of many of our corporate tools), and one running Fedora Linux (where I do some open source development).

ctrl-alt-del shuts down guest OS
I am using a windows host and I always lock the screen when I leave my desk/computer. Sometimes I happen to be in my linux VM at the time and this causes the linux system to log me out and/or shutdown, neither of which I appreciate, especially if I have lots of work in progress. I could figure out how to stop this within Linux, but I really just want VMWare to ignore the ctrl-alt-del and let me send one explicitly there if I need to.

I achieved this by adding the line:
```
mks.ctlAltDel.ignore = "TRUE"
```
to the "C:\Users\All Users\VMware\VMware Workstation\config.ini" file. This tells the VMWare to ignore the ctrl-alt-del and so the client's don't see it. If I want to send a ctrl-alt-del to the client, I use the VMWare defined ctrl-alt-ins combo.
Shared Folders are slow in Windows XP Guest
My Windows XP guest was extremely slow in accessing shared folders (to the point that I didn't want to use them). At first I just thought this was normal, but then after a quick google search, I found this :
```
1) Create a text file called 'lmhosts' in the folder 
   C:\WINDOWS\system32\drivers\etc - if it doesn't already 
   exist. If it does, simply edit it.
2) Add the following line:

        127.0.0.1   ".host"

3) Save the file. 
```
This is done in the Guest OS and it worked like a charm, though I didn't consistently have the slowness problem before implementing this and didn't study it long enough to figure out the specific mixture of circumstances to cause the problem. Implementing this fix got rid of the problem in all situations (so far).
Text input cursor icon disappears in Win XP guest
In my windows XP guest, the standard I-Beam text input mouse cursor icon (the one that is used when the mouse is over a text input field such as a field entry, or an edit box) would not show up. I would be left without an indication of where my cursor was. At first, with just editing some forms, this was just an annoyance, but later, when I went to edit a document or an email message, it was downright painful.

I first tried fixing this by changing the cursor icon. This worked in some cases, but left the most important (editing docs/emails) still broken. Some more searching (and this took a bit of work) and I found the right article in VMWare's forum which included:
```
In the guest, try dropping the display hardware acceleration down a notch.

Start->Settings->Control Panel->Display
Settings->Advanced->Troubleshoot->Hardware acceleration
```
Note that there's also a "Troubleshoot" button on the Settings Tab. This isn't the one, you want to use the Advanced button and then go to the Troubleshoot tab.

For me, dropping it down one notch (to turn off some of the acceleration of the cursor operations) was all that was needed.
Printing from a Windows XP Guest
Printing from my Windows XP host was a problem as I would sometimes be connected to the corporate VPN and sometimes not. While on the VPN, the printers on the system's local network were not available from the guest as the connection to the physical network was through a NATed VMNet and thus two levels away from the guest.
I worked around this by sharing the printer from my host OS and then using the host-host VMNet to access that "network" printer -- which was a local connection and thus allowed under our VPN configuration. This works whether or not the VPN is up and running.
Conflict between Communicator and VMNet setup
In my Windows XP guest, I was unable to connect to our company's Microsoft Office Communicator SIP server. Playing around with this for a while, I was able to determine that the problem was related to my host-only VMNet. Disabling the VMNet allowed Communicator to connect, enable it and Communicator would again fail to connect.
The problem was that the DHCP server was setting a DNS server in the guest host and the failure of that DNS host was causing the problems (probably timing) for Communicator. So, I disabled DHCP on that connection and hard-coded an IP address for the host and guest OSs manually and did *not* specify DNS servers for that connection (didn't need them) and voila, it all worked fine.

Tags : VMWare / Vista / Linux / Windows / Vista / XP / tricks

Derived trust

2007-07-08T05:27:00.000-07:00

Eric Norman, commenting on my chastization of Chase asks me:

Do you have any idea about what your mother would have an easy time of? That is, your mother would be able to say, "Yes, this is my bank", or "Wait a minute; something is wrong here" and get the right answer every time.

Would the green address bar be enough for your mother?

I started to answer in a comment myself, then thought that this topic was important enough to require its own discussion topic.

The answer to the "green address bar" being enough, of course, is: No. Color, pretty locks, etc. would not be enough for my mother nor, I suspect, many other mothers, fathers, brothers, sisters, etc.

What my mother needs is a means of deriving the trust of a site from other people that she knows and trusts and to have any site that isn't on that list to either be totally blocked or to set off all kinds of bells and whistles so that it's impossible for her to not realize she's walked out of the nice safe world into the dark inner city of the internet.

My mother would trust sites that I, or probably most of my siblings, had said were OK (which is essentially how she does things today, but with a phone call and without protection within the platform that she really is looking at the actual site one of us said was OK).

This would require some client enhancements in browsers and possibly in mailers, some reputation based host that she could point her client towards to say "include Conor's list in my set of sites," a means to get real-time approval, support for multiple such lists (so she could include my sister's list, or my brother's list) etc. etc. I think she would set it to block any non-OKed sites. Others would probably want to be able to add their own sites as well.

As I think about this, much of it feels like the kind of infrastructure AOL has in place for their parental controls (where the parent can control what their youngster has access to), though this would be the reverse direction and rather than a control, it would be advisory (because my mother could change the settings on her browser and do whatever she wants on her computer).

Tags : trust / safety / phishing / AOL / parental controls / reputation

The Vast Machine

2007-07-07T18:42:00.000-07:00

On my last trip, I picked up a book at one of the airport bookstores. The Traveler by John Twelve Hawks. It's a good story about the struggle between good and evil and I recommend reading it.

The reason I bring it up here is because it paints a pretty strong fictional picture of what could be done by the wrong hands in our ever-more-connected world. They called it "The Vast Machine" and fictionalized how the bad guys were able to tie together information from every kind of source to create a super surveillance system capable of finding anybody who even touches the grid. Using ATM video feeds to track a victim, using toll boot cameras to track cars, planting false criminal records to get law enforcement to do their work, etc., etc..

On of the memorable sequences discussing the US's choice to put RFIDs into passports (supported and driven by the bad guys, of course):

"Is the information encrypted?" Michael asked
"Of course not. That would make it difficult to share the technology with other governments".

"But what if terrorists use the skimmers?"

"It would certainly make their job easier. Let's say a tourist was walking through the marketplace in Cairo. A skimmer could read his passport -- find out if he was an American and if he had visited Israel. By the time the American reached the end of the street, an assassin could be stpping out of a nearby doorway."

Michael sat for a moment and studied Nash's bland smile. "None of this makes sense. The government says it wants to protect us, but it's doing something that makes us more vulnerable."

General Nash looked as if his favorite nephew had just made an innocent mistake. "Yes, it's unfortunate. But you have to weigh the loss of a few lives against the power given to us by this new technology. This is the future, Michael. No one can stop it. In a few years, it won't just be passports. Everyone will carry a Protective Link device that tracks them all the time."

Scary. Very scary. Fictional yes, but not outside the realm of possibilities given current or near future technologies.

This certainly reinforces the need to study the long term privacy impacts of all this magical work we're doing in the Identity space and especially with the move to contactless transactions.

Anyway, good summer reading for everyone and especially for those in the identity space.

Tags : identity / privacy / The traveler / RFID

They Just Don't Get It....

2007-07-07T08:04:00.000-07:00

Received this email the other day from Chase (the banking folks who are frequent targets of phishing attacks).

I'm still amazed that financial institutions continue to send emails to their customers with active hyperlinks and directions to use those links. This encourages the exact behavior that makes their customers susceptible to a phishing attempt. After checking the links closely (I do like to study phishing attacks) as well as the rest of the content of the message, the only thing that provided any evidence to me that this was actually from Chase was the 4 digit portion of the account number (something buried deep down in the message).

What's especially interesting in this case is that I have already used their online payment system to make the payment for the current statement, so they are sending me an email to tell me to use a link to do something that I've already done.

We need to move away from these kinds of emails until there is some way for the average user to authenticate that they came from the real party with which they have a relationship with and not some phishing impostor. Yes, I can tell verify this because I'm the suspicious type but my mother would have a hard time with it.

Tags : identity / phishing / identity theft / chase

Equipment recycling

2007-07-05T15:58:00.000-07:00

In all my collecting of gadgets and toys, I've built up a collection of unused electronics. Thank god for eBay as I'm usually able to get rid of working electronics without too much trouble (in fact I think Ebay has created an entire new model for gadget upgrading and trickle down flow as us gadgeteers sell our gadgets as soon as the next gadget comes out).

Of course, I also have stuff that just isn't worth selling on eBay -- usually because it is broken or the shipping cost is just too high when compared to the value of the item. This is especially the case for old computer monitors.

Well, today, I looked around the house and found 5 monitors ranging in size from 13 inch to 21 inch (the 21 inch Viewsonic was dead, the others work, but were old and unused), a 27 inch Sony television, and a dead APC rack mount UPS (when I called APC about it like a year ago, they said that "it gave its life to protect all the equipment behind it"... I thought that was a lame way of saying "we don't want to pay to fix it, you're on your own").

A quick search for electronic recycling found E Tech Recycling which, interestingly has two offices in the U.S. -- One in Hillsboro, OR (where my work office is) and one in Chantilly, VA (close to where I live). We loaded the stuff into my wife's car (it was raining and I didn't think it would be wise to put electronics into the back of my pickup truck) and drove off to the local E Tech.

They helped unload the car, and charged me $65 -- which, I think, was a very good deal for everyone.

Tags : electronics / recycling / green / eBay / Viewsonic / APC

Cygwin on Microsoft Vista

2007-07-04T18:26:00.000-07:00

I've run into a few problems using Cygwin on Microsoft Vista on my new laptop:

The installation hung during the post-install step "/etc/postinstall/gnuplot.sh". Researching this problem on the google found an article on the cygwin mailing list which worked around the problem by setting the Windows XP compatibility mode on the installation executable. I wasn't comfortable with this solution as it might cause things to not work well later on Vista.

So I poked around a bit, turned on the "Command Line" output for the "Processes" tab in the Windows Task Manager and found that the script was hanging on "/usr/bin/texhash", which rebuilds a directory listing used by Tex. I was able to run the command successfully on the cygwin command line, but it was still hung in the installation process. So I used Task Manager to kill the texhash process and the installation continued to a later step "postinstall-ec-fonts-mftraced.sh" where it was trying to run "mktexlsr" - which is the same program. I again used Task Manager to kill that process and the installation now went on to a succesfull completion.

Following the install, I went back and ran the texhash program manually, which did require me to change the mode of the files "/var/cache/fonts/ls-R" and "/usr/share/texmf/ls-R" files which had been left read-only when I killed the process.

Everything seems to be working fine now.
Scripts moved over from my Windows XP installation of cygwin now fail to run because they are DOS formated (\r\n line termination vs UNIX's \n). I don't recall setting anything special when I installed cygwin on my old system, but on the new install, it clearly asked if I wanted to use the binary (UNIX) mode vs DOS mode and recommended Binary (which I picked). Not a big deal... Just ran "tr -d '\015' < file > file.new" for any such scripts to get rid of the \r's
My rsync backup scripts failed with strange errors and paths. I poked around a bit and this seems to be caused by the extensive use of NTFS's Junction points within the c:\users\user_name profile directory (including a particularly problematic one where "c:\users\user_name\AppData\Local\Application Data" points back to "c:\users\user_name\AppData\Local" creating an infinite loop, even for Windows Explorer (you can infinitely open "Application Data" again and again going as deep as you want since at every stage you get the contents of "Local" again which has the "Application Data" junction again within it).

Junction points look like a shortcut in Windows Explorer and behave like a Unix symbolic link to some extent. I couldn't find anything within the cygwin mailing lists, nor in the rsync man page to deal with this problem, so I just manually excluded the problematic entries from the backup set and things worked fine.

Tags : Cygwin / Vista

Gadget of the Week #12

2007-06-30T12:36:00.001-07:00

My latest gadget is my new Dell Latitude D830. This replaces my older Dell Precision M70 and includes all the available bells and whistles. 2.4GHz Intel Core 2 Duo processor, 160GB 7200RPM drive, 4GB memory, 802-11a, g, and *n*, thumbprint reader, TPM, theoretically long battery power, and, of course, Windows Vista Ultimate. It is also thinner and lighter than my former laptop (and the Dell latitude D810's we have here).

The battery life is nowhere near the claimed "up to 9 hours" (which I didn't expect it to be, given how I use the system), but it does last about twice as long as my former laptop with the same relative workload -- about 5 hours now, easily going the entire cross-country flight either direction which I tested last week. With the old laptop I had to bring along a spare battery and used them both up pretty well on the same trips (or I brought a power adaptor).

This is a screamer of a system. About the only thing I can say negative about it is that it only came with the integrated graphics card (they did not offer an option for an enhanced graphics card at the time I ordered -- they do now, but there doesn't seem to be an option to add it to an existing system).

So far Microsoft Vista has been OK. There are some things I like (recent places). There are some things that I miss from Windows XP (hardware profiles being one of them). I'll make a separate report later on Vista as I get more used to it and figure out the tricks.

One other note of interest: While the system does have 4GB of memory installed, the fact that I am running a 32 bit operating system (Vista) which can only address a total of 4GB and has memory reserved for hardware i/o mapping (and some shared memory for the graphics card), the net amount of memory I have is around 3.4GB. Much lower than I thought. I'm thinking about making the jump to a 64 bit OS next time.

I did come like this close (picture me with my thumb and index finger almost touching) to jumping on the Macbook Pro bandwagon this time. I just couldn't give up the 1900x1200 display and the availability of real docking stations. Perhaps next time.

As part of this upgrade, I have done my part for the tech economy, choosing to buy/install upgraded versions of most of my like 5 billion utilities that I use. I guess Microsoft does cut a wide economic swath within the tech industry.

This is the first system upgrade in more than 2 years for me and it was worth it. I've been looking for a while and waited for Intel's Santa Rosa platform to become available.

Tags : gadget / Dell / Latitude / D830 / Microsoft / vista / intel / Core 2 Duo / MacBook Pro / Santa Rosa

You know you're an addict when....

2007-06-25T14:00:00.000-07:00

Back in February, I wrote about getting hooked up with Where's George after finding a dollar bill in my change with some strange markings on it.

I quickly went out and got my stamps and started marking some bills.

Well, 4 months later (today), I am totally hooked on it. So far I have:

entered 727 bills worth a total of $5,177 (many of them are $1s).
gotten 44 hits on my bills, most of them coming in the last month or so (seems you need to build a certain amount of inertia before the hits start rolling in)
achieved a "George Score" of 637.36, placing me above the 85th percentile of Where's George users!!!!!
joined Friends of George where I get to pay extra money for the joy of entering my bills and tracking them :-).
started paying cash for many transactions that I had been paying with credit cards (so I can mark more bills and cause money to flow).
learned that it is better to mark small bills ($10s, $5s, and $1s) as they circulate alot more than $20s, $50s, and $100s which frequently just go to the bank awaiting a future withdrawal.
learned that it is better to get a stamp that needs little to no alignment (rather than the nice circular stamp I bought that goes around the treasury seal and must be aligned somewhat carefully -- taking way too much time when trying to stamp a stack of 100 or 200 $1s).

The real signs of my addiction:

The anticipation that I have when checking emails looking for hits.
The groans that I hear from my "friends" when I pull out my wad of marked bills trying to exchange them for unmarked bills in their pockets.

In any case, I still find it lots of fun and my friends seem to be mostly amused with my addiction. If you're interested, go get a stamp and enjoy marking your bills!

Tags : Wheres George / money

Perhaps not so much Bashing...

2007-06-24T13:37:00.000-07:00

Kim responds to my note about SAML Bashing:

...which is, by the way, absolutely NOT my intent. I’m simply trying to understand how SAML relates to linkability, as I am doing for all the other major identity technologies. I can’t take up all the points he raises, but encourage the reader to look at his piece…

Perhaps I reacted too negatively, but the analogy of some unknown clerk reaching into my pocket really irked me as that clearly isn't what happens and it appears to be written to instill unreasonable fear in an implementation of a browser-based SSO protocol.

I’m not criticizing or discussing the profile for an Enabled Client/Proxy. I was talking about SAML as we know it - in the mode which has been widely deployed in portals all over the world.

I think such analysis should be based upon the capabilities of the protocol and not about what some deployments have chosen to do within their environment (where they clearly felt that browser-based SSO meets their needs (and in many cases is mandated by the deployment scenario).

I think Conor is misunderstanding my intentions. I agree that with a completely trustworthy Identity Provider following best practices for end user privacy, Conor’s b) and c) above would apply. But we are looking at linkability precisely to judge the threats in the case that parties to identity transactions are NOT completely trustworthy (or are attacked in ways that undermine their trustworthiness.) So arguing that the identity provider will behave properly has nothing to do with what I am exploring: risk. I’ll try to build Conor’s concerns into my ongoing discussion.

I'm sure there's some misunderstanding here. I normally find that I agree with most of what Kim has to say and really respect his opinions.

As far as the trustworthiness is concerned, there's nothing that is completely trustworthy, not even if I make the decisions myself and hand-code the response messages from the keyboard (I'm sure that I will make mistakes of judgment and or typos).

I would ad that the same "attacked in ways that undermine their trustworthiness" applies to client implementations that try to enhance privacy protection. They too are subject to being attacked. Nothing is totally foolproof and I'm not sure which has more likelihood of successful attack, a service maintained under contractual agreements or open software systems in the hands of end user.

I certainly have chosen to put my money in a bank rather than store it under my mattress. Yes, the bank is more likely a target for a robbery, but they are legally obligated to maintain my funds, even if they are robbed. Similar decisions will be made by many people in the identity space (and yes, some out there will always keep their funds in their mattress).

Tags : identity / privacy / SAML / CardSpace / liberty

SAML Bashing

2007-06-24T08:23:00.000-07:00

Kim writes about SAML's use of redirection protocols.. To start with, he forgets to mention a few important facts as part of his discussion:

SAML defines a profile for an Enabled Client/Proxy (ECP) which is an evolution of the Liberty Alliance's LECP protocol. This protocol does *NOT* involve redirection, but instead supports an intelligent client directed by the user driving SSO transactions (a similar model to that adopted by Cardspace).
The Browser-Profile that Kim is referring to is one written based upon a use case requirement that the profile work out-of-the-box on unmodified browsers. There is NO other possible solution that will work in this scenario that will protect the users credentials at the IdP.

That said, there are still several statements in Kim's analysis that I feel obligated to respond to. These include:

Note that all of this can occur without the user being aware that anything has happened or having to take any action. For example, the user might have a cookie that identifies her to her identity provider. Then if she is sent through steps 2) to 4), she will likely see nothing but a little flicker in her status bar as different addresses flash by. (This is why I often compare redirection to a world where, when you enter a store to buy something, the sales clerk reaches into your pocket, pulls out your wallet and debits your credit card without you knowing what is going on. (”Trust us.”)

First off, the user only see's nothing if a) they are already authenticated by the IdP, b) they have previously established a federation with the relying party, and c) they have told the IdP that they don't want to be notified when an SSO with this party takes place. I, for one, want things to work this way for me with providers that I trust (and yes, I do trust some providers). The inability to do this type of automatic operation is one of the shortcomings in Cardspace's implementation that I think will eventually be fixed. There is no need to have repeated confirmations of operations that I say may occur without my unnecessary participation.

Secondly, the analogy is way off base, trying to make this seem like I'm bing pick-pocketed by someone I don't know which Kim knows is absolutely not the case. A more proper analogy would be something along the lines of "I give one of my providers permission to reach into my bank account and withdraw money to pay my bill". I do this all the with providers I trust, such as my electric company, my telephone company (both wired and wireless) and may other companies.

So, returning to the axes for linkability that we set up in Evolving Technology for Better Privacy, we see that from an identity point of view, the identity provider “sees all” - without the requirement for any collusion. Knowing each other’s identity, the relying party and the identity provider can, in the absence of appropriate policy and suitable auditing, exchange any information they want, either through the redirection channel, or through a “back channel” that dispenses with the user and her browser altogether.

The IdP does not "see all". The IdP only sees that you have visited a particular relying party. It does not see what you do at the relying party. Knowing that I visited Amazon, is not the same thing as knowing what I looked at and/or purchased at Amazon.

Secondly, my choice of an IdP (as with most others) would be made based upon the appropriate policies and auditing capabilities at that IdP (just like I don't choose to use Johnny down the block as my bank, I choose a reputable firm and just as I would require the exact same policies and auditing in any client that I chose to use to act as my identity selector (yes, I have to *trust* Cardspace's or Credentica's implementation of policies just as I have to trust an IdP's).

In fact all versions of SAML include an “artifact” binding intended to facilitate this. The intention of this mechanism is that only a “handle” need be exchanged through the browser redirection channel, with the assumption that the IP and RP can then hook up and use the handle to “collaborate” about the user without her participation.

That isn't the intention at all. The intention, as Kim surely knows, is to pass a message by reference rather than by value. For the non-programmers in the audience, this means that I have a message that I need to send to the relying party (in this case that message contains an assertion, which can be big and complex and which has additional security requirements if passed through someone else's hands -- yes, the user can count as someone else). Instead off sending the token to the client to have the client then send it up to the relying party, I can send a small artifact that the relying party then presents to the IdP to get the token. The protocols explicitly define what the artifact is exchanged for -- it was never intended as, nor can it be used within the protocol definitions, as a general collaboration handle.

In many enterprise implementations, the artifact is used to allow the IdP to issue assertions to the Relying Party that don't need to be signed by the IdP. Clearly that isn't something I could do if the assertion was sent to the client (otherwise we'd be talking about how I took the token and edited it say I was Bill Gates when I sent it to his bank).

In considering the use cases for which SAML was designed, it is important to remember that redirection was not originally designed to put the “user at the center”, but rather was “intended for cases in which the SAML requester and responder need to communicate using an HTTP user agent… for example, if the communicating parties do not share a direct path of communication.” In other words, an IP/RP collaboration use case.

All SSO use cases (where one party authenticates a user and asserts an identity for that user at a relying party), redirection or not, would then be, by Kim's definition, an IdP/RP collaboration since the RP (Relying Party) is relying on the identity presented by the IdP. This has nothing to do with redirection or user involvement, or SAML in particular.

As Paul Masden reminded us in a recent comment, SAML 2.0 introduced a new element called RelayState that provides another means for synchronizing or exchanging information between the identity provider and the relying party; again, this demonstrates the great amount of trust a user must place in a SAML identity provider.

No. RelayState is designed for the RP to send information to itself, not the IdP, so that it can remember what the user was trying to access when the user is returned to the RP following a successful SSO operation. This is primarily used in the case where the RP is unable to set a cookie in the user's browser to remember that information. SAML even points out that as little as possible data should be included in the RelayState.

Paul's point in his comment was that if an RP used this incorrectly, they could leak information. The SAML specs contain exactly this caution.

I don't claim to say that SAML is the end-all for every use case. I do believe that we need to support multiple methods, some of which have different privacy implications. I also don't want some privacy weenies making life intolerable by the need for a confirmation of every thing that I already said it was OK to do. I do trust some of the parties that I interact with and want to be able to automate as much as I feel comfortable doing. I have no problem with the privacy weenie that wants to turn on the "let me approve everything" -- just don't force me to live that way as well.

Tags : identity / privacy / SAML / CardSpace / liberty

You know you're late when...

2007-06-23T05:31:00.000-07:00

Earlier this week, a computer glitch in United's sysytem caused hundreds of flights to be delayed or canceled. As my luck would have it, I was caught up in the delays. My flight was one of the last flights from San Francisco to Portland and was supposed to arrive at 11:59 that night. The arrival was delayed until around 2:15 AM and I didn't arrive at my hotel until after 3AM.

This was made worse by the fact that the previous night, my flight from Dulles to San Francisco was delayed by close to 3 1/2 hours by thunderstorms in the Dulles area (causing that 5 hour flight to be 8 1/2 hours as they didn't start the ground hold until we were already loaded onto the plane). Luckily, it was an internationally configured 777 and I had upgraded into business class, so at least it was comfortable.

I really knew I was late getting to my room at the hotel when I found that my USA Today for the next day was already delivered to the room.

Tags : travel / UAL / United / dulles / IAD / SFO / PDX

Updated Liberty Open Source

2007-06-20T06:24:00.001-07:00

I've updated my Liberty ID-WSF Open Source implementation for both the server side and client side to include support for much of the the Advanced Client Provisioning Service specification.

The Advanced Client Technologies Overview is a good starting point for understanding what we're trying to do with the advanced client specs.

This release also includes some updates/fixes in the basis ID-WSF support.

Have fun!

Tags : identity / liberty / Liberty Alliance / Open Source / ID-WSF / Provisioning / Advanced Client

Concordia Schmordia

2007-06-19T07:30:00.000-07:00

Next week, I will sit on a panel with Mike Jones of Microsoft and David Recordon of VeriSign. This panel will be a part of day spent on the subject of the Concordia Project.

One might ask, "What's Concordia?" and I, of course, would respond that that's a good question. My recollection (questionable, I know) of the sequence of how we got to where we are is as follows: The name originated in an earlier internal project at the Liberty Alliance where a number of us were examining potential paths towards convergence with the other technologies/protocols in the identity and web services space. Eve Maler can be blamed for the name as she brought it up given that Concordia was the Roman goddess of agreement, understanding, and marital harmony (not that any of us were getting married to each other) -- which, theoretically, is what convergence is about.

Anyway, as we moved forward on the project we eventually figured out that doing this ourselves would likely be a waste of time (why would anyone else listen to us). If we really wanted to talk about convergence we needed to bring the other players to the table. That led to more deep thinking (and perhaps a few visits to the neighborhood psychologist) and the eventual realization that many of the differences in the current approaches had to do with different use cases and with looking at the problem from different points of view.

So, here we are. Trying to organize an effort to bring together people interested in this space so that we can discuss our respective use cases and understand the problems that need to be solved. Liberty has tried to be very careful and very clear that this isn't a Liberty effort, but an industry effort (which Liberty supports). My hope is that we can use this common understanding to drive towards common protocols, features and capabilities so that those trying to use our stuff will have an easier time integrating with the rest of the world.

So come join us for the Concordia day at the Burton Catalyst Conference. At the minimum, it should be fun (though without Dick Hardt on the panel, I won't have someone to pick on :-)). The Concordia sessions are free, you just need to register here.

I look forward to seeing you all there!

Tags : identity / Liberty / Liberty Alliance / Microsoft / Verisign / Burton Catalyst / Concordia

Business traveler Magazine

2007-06-18T17:45:00.000-07:00

Partly from some of my writings here about my frequent trips on United, I was interviewed for this article in Business Traveler Magazine. Of course, the picture in the article was not a picture of me -- I think it might be Britta, there's some resemblance or, perhaps, that's just wishful thinking :-).

I should also point out that I wasn't the one to find the article. My sister, Theresa, was looking for our family web page (where I put up family photos) on google and she found the article and told me about it. Yes, I knew it might be coming at some point, but wasn't aware it was published. Now if I could only get a print copy to hang onto :-).

Tags : United / travel / Global Services / Business Traveler

Annual Credit Reports

2007-06-18T07:48:00.001-07:00

Not sure what brought this up now, but for some reason I thought to go get the "free" annual credit reports that we are entitled to here in the US at least once every year. I started with AnnualCreditReport.com (the central clearing house setup by the 3 major credit reporting agencies) which asked me for the standard credit information (name, ssan, address, dob, etc.).

Once I entered that information they prompted me to select which of the 3 agencies I would like to review the report at (yes, you can select multiple). I selected all 3, but now, after thinking about it, perhaps I should have selected one only and rotated around the 3 agences throughout the year -- that way I can get a new checkup every 4 months rather than only one per year -- Oh well).

The web site then forwards you to each of the agencies where you will go through a different process at each of them to further verify your identity and get access to your credit report. My experience at each of them:

The first agency I was sent to was TransUnion where I had to create an account in order to get my report. I was then prompted for 2 account numbers (which I had to go find) and then the tried to upsell me a copy of my credit score for $7.95 before they would let me see my credit report. When I was done, the process had taken so $#@%ing long that my session at AnnualCreditReport.com had expired and I had to start all over for the remaining two agencies.
On to Equifax, where they wanted to know which provider I opened an account with in 2005 and what was the payment (much easier to deal with), then they too wanted to upsell me my credit score for $7.95. No need to create an account. Got my report and printed it.
And, finally, Experian ((the last of the 3). This time, I went back to AnnualCreditReport.com fairly quickly, so no need to re-enter all of my information. Experian started with verification of my ssan (last 4 digits), then they asked for verification of 2 accounts (who gave me a mortgage at a particular time and who gave me an auto loan at a different particular time) as well as the name of the county in which I live. Poof. I got to see my credit report. No need to create an account, no attempt to upsell me with the credit score (at least not as an intrusive click-through step -- it may have been there somewhere else on the page that I just ignored).

Moral of the story.... be tenacious and make them give you a copy of your report -- it IS free (as long as you don't fall for the upsell). Unless you think you have been a victim of identity theft, I would stagger the reports from each agency as many of them carry the same information as the others and this kind of gives you better coverage over the year (there's a lot someone can do in the 12 months between your annual reviews if you do all 3 at the same time).

Tags : identity theft / credit report / free credit report / Experian / TransUnion / Eqiuifax / AnnualCreditReport.com

The Combo

2007-06-04T09:58:00.000-07:00

Paul must be getting a little hard up for cash nowadays as he seems to be out moonlighting.