Ben Laurie (yes, *that* Ben Laurie) reports that TLS is flat busted.

For the last 6 weeks or so, a bunch of us have been working on a really serious issue in SSL. In short, a man-in-the-middle can use SSL renegotiation to inject an arbitrary prefix into any SSL session, undetected by either end.

To make matters even worse, through a piece of (in retrospect) incredibly bad design, HTTP servers will, under some circumstances, replay that arbitrary prefix in a new authentication context. For example, this is what happens if you configure Apache to require client certificates for one directory but not another. Once it emerges that your request is for a protected directory, a renegotiation will occur to obtain the appropriate client certificate, and then the original request (i.e. the stuff from the bad guy) gets replayed as if it had been authenticated by the client certificate. But it hasn’t.

More here.

There are three general attacks against HTTPS discussed here, each with slightly different characteristics, all of which yield the same result: the attacker is able to execute an HTTP transaction of his choice, authenticated by a legitimate user (the victim of the MITM attack.

Overheard on die.net’s jabber server: “It basically says that since Gmail et al are in “possession” of your e-mail, a warrant only needs to be served on them, and it can include a provision that they aren’t allowed to notify the owner.”  (‘it’ being this decision, blogged about here and here)

From the decision:

Thus subscribers are, or should be, aware that their personal information and the contents of their online communications are accessible to the ISP and its employees and can be shared with the government under the appropriate circumstances. Much of the reluctance to apply traditional notions of third party disclosure to the e-mail context seems to stem from a fundamental misunderstanding of the lack of privacy we all have in our e-mails. Some people seem to think that they are as private as letters, phone calls, or journal entries. The blunt fact is, they are not.

District Judge Mosman seems to be making an interesting conflation that I find… odd… for a judge to make: “because things technically are a certain way, people should know that they are that way, but… regardless of whether they know it or not, we should treat them that way.”  Why should people know that they are that way?  We expect people to understand the full implications of Internet RFCs and how they interact?  Why should we assume that?  Even if we could assume that, there’s lots of other technical implications that can lead to abuse of legal authority, and those are curtailed on different rationales, why should we treat email differently?

There are a number of inconvenient blunt facts, Judge Mosman.  It’s a blunt fact that your cell phone tracks your whereabouts.  It’s a blunt fact that many people’s cars track their whereabouts.  It’s a blunt fact that I can stand up a laser and a few other bits of technology that are commercially available in the United States, point them at my neighbor’s curtains, and get a nice full blown real audio copy of their conversations (this one’s pricey, but here just for reference that I’m not making that last statement up).

What prevents misuse of these technologies is the mechanism of legal warrant: if someone wants access to this sort of data, they’re required to convince a judge that this information is pertinent to a legal action, get said warrant written, and execute it.  Now, *physical* searches are different from wiretaps; this has long been recognized by courts.  One can argue that monitoring someone’s email is monitoring communications, and thus should be treated more like a wiretap than a physical search.  When you go to court to get permission to wiretap a target, all you have to do is exectute that court order on the telecommunications company, you don’t have to inform the target.  You can also get a warrant for someone’s telephone records, but you don’t have to inform the target.  In that light, it doesn’t seem entirely different from the issue of email, right?

But email *is* different from telephone communications, because you’re not just getting someone’s *current* communications, you’re getting someone’s data store.  This isn’t like someone’s phone records, which is just a call log, this is like having retroactive access to everyone’s phone conversations, backwards in time.

There’s a quantifiable and quantifiable difference here.  Certainly, the question of whether or not the government should be able to execute a warrant on someone’s email store is an outstanding legal question, but “well, it can be done and people should know that it can be done so they can’t expect privacy” seems like a really weak sauce position here…

… and coincidentally, here’s a case where the ruling was found to be precisely the reverse…

In an interesting side note, a federal judge ruled yesterday that jurors in the Bear Stearns case (in which Cioffi and Tannin are accused of making their portfolios sound much healthier than they were) will not be permitted to hear about one email, in which Tannin wrote, “I became very worried very quickly. Credit is only deteriorating. I was worried that this would all end badly and that I would have to look for work.”

Judge Frederic Block ruled that the government’s search warrant filed with Google to obtain access to the e-mail was unconstitutionally broad and “did not comply with the Warrants Clause of the Fourth Amendment.”

What’s the moral of the story?  Well, it’s certainly the case that outsourcing providers have a bunch of your data.  It’s also certainly the case that it’s not currently stated in law what the legal obligations are of such an outsourcing provider vis-a-vis protecting your data from nefariousness, let alone government subpoenas or warrants (nor is there a suitable body of case law).  Heck, many outsourcing providers aren’t even in this country, so there’s no guarantee that even *if* legislation is drawn up (or case law reaches some preponderance of decisions) that you’re going to have U.S. legal protections over your data.

It just means that you ought to keep this sort of thing in mind, when you’re deciding whether or not to outsource…

As promised, albeit a bit later than originally scheduled, the second part of “How To Hire A Sysadmin”, with the Second Question to ask a potential hire:

“You are sitting at your desk one day when the Chief Operations Officer shows up out of nowhere and says, ‘I believe that my assistant Frank has been communicating company secrets to competitors.  We let Frank go this morning.  I want a copy of all Frank’s files and his email put into the Operations share on the file server so that the Ops group can go through it to see what the damage is.’  What do you say?”

I’ll give you a hint, the best answer is a very quick and unequivocal “I’m sorry, Dave, I can’t do that.“  The second (almost as acceptable) reply is, “I’m afraid you’ll have to have that okayed by the office of legal counsel.”  [edited to add]  This actually can go either way in terms of what’s the “best” answer.  I’m getting ahead of myself.

I’ll give you another hint, hiring someone who answers the question correctly may turn out to be a pain in your butt later.  It’s for your own good.

In spite of the fact that (at least in the State of California) your employees’ email accounts are regarded as property of the company, the fact still remains that you (in this case The Company or The Organization or whatever) have legal responsibilities when it comes to employee data.  From Harvard Law (just one example):

The general idea has been that the employer owns the equipment, and can therefore set the terms of its use. Even under current law, which has been deferential to employer monitoring, this does not mean that employers are free to monitor or not monitor at will. It is not clear, for example, whether employers who fail to notify their employees that they monitor their mouseclicks will avoid liability for invasion of privacy. Moreover, even if employers issue a general notice to employees that they may be monitored, an employee might argue that more specific notice is required.

Even if your workplace has very well defined rules regarding employee’s use of electronic equipment, be aware that you can have all kids of serious repercussions for mishandling data.  If Frank’s mother recently emailed Frank that she had Hepatitis C, and that information winds up getting out, you can be up for a world of painola.

Handling data discovery, even internal to the company, is not something that a systems administrator should *ever* regard as his (or her) blanket responsibility.  They shouldn’t be monitoring general computer usage either, but that’s a subject for another post.  Oh, wait, I already wrote that one.

In the United States, there are three classical professions, namely… doctor, lawyer, and priest according to Wikipedia.  It’s generally given now that this list is somewhat longer, the Department of Labor has a whole list of “professional and related occupations“.

Generally, to be officially regarded as a “profession” (as opposed to an “occupation”) you need to meet a few requirements, one of which is some sort of professional body.  Borrowing straight from the Wikipedia page:

Professions are typically regulated by statute, with the responsibilities of enforcement delegated to respective professional bodies, whose function is to define, promote, oversee, support and regulate the affairs of its members. These bodies are responsible for the licensure of professionals, and may additionally set examinations of competence and enforce adherence to an ethical code of practice. However, they all require that the individual hold at least a first professional degree before licensure. There may be several such bodies for one profession in a single country, an example being the ten accountancy bodies (ACCA, ICAEW, ICAI, ICAS, CIMA, CIPFA, AAPA, CIMA, IFA, CPA) of the United Kingdom, all of which have been given a Royal Charter although not necessarily considered to hold equivalent-level qualifications.

Typically, individuals are required by law to be qualified by a local professional body before they are permitted to practice in that profession. However, in some countries, individuals may not be required by law to be qualified by such a professional body in order to practice, as is the case for accountancy in the United Kingdom (except for auditing and insolvency work which legally require qualification by a professional body). In such cases, qualification by the professional bodies is effectively still considered a prerequisite to practice as most employers and clients stipulate that the individual hold such qualifications before hiring their services.

There is no nationally-recognized “systems administrator” professional body (there ought to be), and yet we have administrative or root access to the file servers upon which sit not just your corporate data, financials, customer lists, etc., but also *your* email and *your* documents (that’s one major reason why there ought to be).  If you are a doctor, a lawyer, an engineer, or an accountant you may actually *lose* the ability to do your job in perpetuity if you fail to exercise your professional responsibilities in the eyes of your professional body; you can lose your license to practice law, or medicine, or submit plans to City Hall, etc.  This is a double-edged sword: if you mess up, you can literally be ejected from the profession.  However, with it comes a protection that currently systems administrators don’t have: if The Boss tells you to do something that violates your professional ethics, you can tell The Boss, “I’m sorry Dave, I can’t do that, because I could lose my license.”  Right now, sysadmins have the responsibility to protect the data, but we don’t have much in the way of business clout.  You want to hire someone who will actively refuse to let you shoot yourself in the foot, even without that backing.

Trust me.

In case you haven’t heard about Superfreakonomics, it’s due out soon, and there’s a chapter in there where the authors attack the global warming theory on a number of points.

It turns out that they do a very bad job of this, and to top that off they… ah, are a bit loose with their citations.

Now, one of the dead giveaways that someone doesn’t know what they’re talking about is when they actually attribute findings or quotations to people that actively refuted the findings or never, in fact said what they’re claimed to have said…

Interesting side note to the “balloon boy” story: apparently, Richard Heene (father of Falcon) is an end-of-the-worlder.

The lawyer for Robert Thomas, an associate of Richard Heene who says he helped the latter develop various ideas for a TV show, said she thought Heene had become obsessed with a quest for TV fame.  Maybe not for its own sake, she said, but so that he could make a lot of money in a hurry in order to prepare for the upcoming end of the world.  Heene “believes the world is going to end in 2012,” said Thomas’s attorney, Linda Lee.  “Because of that, he wanted to make money quickly, become rich enough to build a bunker or something underground, where he can be safe from the sun exploding.”

Mr. Heene, a quick note: if the sun explodes, burying yourself in the crust of the earth is going to protect you precisely not in the slightest.  This guy had the wrong idea, trying to get on “reality TV”.  He’s a natural Internet Nut.

Cook County state’s attorney office is subpoening the class records of a bunch of Northwestern students involved in The Innocence Project, apparently because they had the gumption to show that the State of Illinois occasionally convicts innocent people.  From the last article:

After spending three years investigating the conviction of a Harvey, Ill., man accused of killing a security guard with a shotgun blast in 1978, journalism students at Northwestern University say they have uncovered new evidence that proves his innocence.

Their efforts helped win a new day in court for Anthony McKinney, who has spent 31 years in prison for the slaying. But as they prepare for that crucial hearing, prosecutors seem to have focused on the students and teacher who led the investigation for the school’s internationally acclaimed Innocence Project.

From another article:

State’s Attorney Anita Alvarez {rejects} the claim that information gathered by students should be protected under the Illinois Reporter’s Privilege Act.

The newspaper says her office is seeking grades and grading criteria, evaluations of student performance, expenses incurred during the inquiry, the syllabus, e-mails, unpublished student memos, and interviews not conducted on the record, or where witnesses weren’t willing to be recorded.

“If you’re going to put yourself into the role of an investigator, then you need to turn over whatever your notes are,” Alvarez said on Tuesday.

Well, now, this isn’t the case whatsoever, Ms. Alvarez.  Even if Illinois Reporter’s Privilege Act doesn’t cover students taking a journalism course (and that requires a huge stretch of logic in there somewhere), it’s simply not the case that you get to demand everything *and* the kitchen sink from these students, as has been pointed out here and here.  Chilling effect aside, it’s simply not true that everything about investigators is regarded as discoverable in a court of law.

All that aside, from a purely professional standpoint… could you possibly choose a *worse* way of presenting a public face to the legal and journalistic community?

This post was supposed to be a follow-up to “How To Hire a Sysadmin”, but I’ve been a little busy studying for a midterm and delving into the capabilities of Alfresco, so I haven’t had a chance to write that post up yet.

In the meantime, this came across my radar from the ISWORLD mailing list and I needed to plunk it somewhere where I wouldn’t forget about it (del.icio.us all too often turning into a pit): Open Knowledge Creation: Improving the Peer Review and Adoption Process.  FTA:

The practice of peer review and acceptance has been in place for many years, predating the Internet, and has recognized shortcomings. The Internet has proven to be a disruptive technology and a means for innovation in many areas of science and society. In this paper we offer an organizing framework aimed at redesigning the peer review and adoption process, referred to as open knowledge creation. The framework proposed utilizes the Internet, Google’s Knol and Groups technology. The open knowledge creation framework consists of four stages: creation, review/revision, evaluation/adoption and publication and is intended to offer journals an alternative for the communication of research that more fully exploits the Internet.

Deserves a thorough read-through and analysis.  Drive-by science bloggers from other fields: what’s your take?

There’s lots of lists out there for “interview questions” to ask IT people when you are interviewing them for a new position.  Many of those lists are pretty worthless in practice, as they actually ask the sorts of questions to which you can find the answers with 60 seconds and a web browser, but they don’t ask the sort of questions that actually tell you anything about the candidate’s capability to understand complex system design.

I really don’t need to know if you’ve memorized the IPv4 header (this is the networking equivalent of memorizing Pi to 40 digits).  I don’t really need to know if you know the difference between the HKEY_CURRENT_CONFIG and HKEY_LOCAL_MACHINE registry hives on a Windows machine, or what the difference is between GRUB and LILO, or what your opinion is of the advantage of the FreeBSD ports collection vs. Linux’s RPMs.  I *really* don’t need to see your Perl coding skills, because if you’re a really good Perl coder you should be writing code, not administering systems.  Not to mention the fact that if you’re administering a lot of systems with home-grown Perl code I probably don’t want to hire you because after 6 months the only person who will have a freaking clue about how the cluster works is the guy who wrote all the tools from scratch in Perl.

What I need to know is if you understand, at a meta-level, what a sysadmin is supposed to do.  You can learn syntax over time (or ask the magic Internet machine).  Learning how to juggle interdependencies is something else.  In fact, quite often those people who are really skilled at syntax (read: recent certification acquisitions) can sound like they really know what they’re doing, without knowing anything about what they *ought* to be doing.

So, I have only two questions for a sysadmin candidate.  Here’s the first one:

“You have a cluster of 300 machines, running 40 different services, on three discrete networks, with two OS-level dependencies. Assuming you’ve built this cluster yourself from scratch with no legacy dependencies, describe this cluster. Feel free to ask as many questions as you like for clarification. Go.”

This is meta-level information mining.  A good sysadmin will spend more time asking questions about what the cluster is supposed to be doing, what sort of services are running, what’s the uptime requirements, who the users are, and what the business continuity requirements look like than they will talking about their design ideas.  A good sysadmin will have a thousand questions.  Note, you have to be able to provide at least theoretical answers to these questions in order to interview a candidate this way.  Second note, if you can’t interview someone this way, you probably should not be involved in the decision making process for new IT hires.

A *really* good sysadmin will ask questions about the physical facility, budgeting, and office politics, not just technology.  They’re going to want to know if they’re going to be able to fix things based upon technological merit, or if there’s a labrynthine approval process that goes through someone who has no technical expertise but absolute veto power over technology decisions… but if you get someone like this in an interview, be forewarned that you’re either hiring someone who will replace your IT manager within 6 months, or someone who will need some other sort of upward mobility within 18 months or they’re going to get bored and go elsewhere.  The price of hiring really great people is that you need to give them really high level work.

We’ll talk about question #2 in the next post.

I’ve been interested in science, mathematics, and philosophy for a long time now.  At various points in my life, I’ve been a practicing mathematician and an armchair philosopher, and I’m currently engaged in scientific research.  I’ve worked for an educational institution specializing in hard core research for almost a decade now.

I’ve known scientists for a long time.  I’ve seen grumpy scientists and happy scientists, conservative scientists and liberal ones (somewhat more of the latter than the former).  I’ve seen scientists have arguments with each other just this short of breaking out firearms and elevating to mayhem, and those same scientists publishing work together after they discovered that they were both wrong about their relative positions and finding the results of their argument to actually be interesting.

I’ve seen lazy scientists, venal scientists, greedy scientists, and scientists with such an odious personal manner that you’d be hard pressed to share a meal with them.  I’ve seen lots of towering egos, and almost as many deflated ones.  Quite a few reconstructed ones in the mix as well.  [edited to add: lest this sound like all scientists are stereotypical difficult personalities, I've also known plenty of nice, friendly, and outgoing scientists.]

I have never seen a scientist actually engage in unethical behavior in their own field.  I know it happens, there are plenty of examples… but on the whole the act of deliberately falsifying results and misrepresenting reality is as uncommon among scientists as running away from a burning building is among firefighters.  Most of them can’t even conceive of the idea; it’s almost incomprehensible.

This is not because scientists are a uniformly ethical crowd (although anyone who will spend their entire life on research that they largely don’t profit from in proportion to the impact of their work is probably going to have at least a high baseline sense of ethics).  It’s because scientists know that reality wins.  If they publish bogus results, sooner or later someone will try to replicate their results, or find some other results that contradict the bogus results.

Scientific argument isn’t like political argument.  The scientific method isn’t even like mathematics.  In mathematics, you declare your axioms and prove interesting stuff follows logically as a result.  In science, you observe reality, make notes, and draw conclusions.  You can have all the nice, logical, consistent theories you want… but when you put a paper up for peer review, or attend a conference, or try to discuss your work with another scientist they’re only interested in the theory in passing.  Reality wins.

What they want to see is the evidence.  Not “beyond reasonable doubt” evidence, but “towering monolithic gargantuan piles of evidence”.  If you don’t have it, you can get eaten alive (at least, metaphorically speaking)… if you don’t have it, you’ve got what is commonly referred to as, “An interesting little theory”, a phrase that itself carries a depth of meaning that isn’t parsed well by people who aren’t learned in the particular field.  The difference between a cap-”T” theory and a little-”t” theory is the difference between the Nobel Prize (not the Peace prize, which has no measurable standard, but the other prizes, that you can only get if you’re the freaking grandmaster ninjitsu tenth-level jedi master gun-kata guru 1,000 lb gorilla of a scientist)… and not getting the invite to go out bar hopping after the keynote speech.

Even if you *do* have a huge pile of evidence, you can get eaten alive if your resulting theory directly clashes with existing theory.  This isn’t because science is hidebound or dogmatic; it’s because scientific theories are based on lots and lots of observations, and if you come up with a new theory that challenges existing theory, you’ve got a pretty high bar to climb over.  People point to all of the major turning points in science as if those moments represented some sort of failure… “See!  They used to think exactly the opposite of what they think now!  They can never make up their minds!”

What those naysayers don’t realize is that “never making up your mind” is a central tenet of being a scientist.  You take some things for granted because nobody has time to learn everything and someone else is better versed than you are, but if someone shows that what you took for granted is wrong, you change your mind.  If you don’t, you don’t get published (at least, not for very long), and that’s the long slow death of the scientist.  Tenure doesn’t mean much if you can’t get a grant.

Reality wins.

Of course, scientific discourse isn’t political discourse.  Scientific discourse isn’t legal discourse.  There’s plenty of studies that show cigarettes cause cancer; it still took decades of fighting misinformation before anyone who worked in the tobacco industry would admit that reality really was described best by the theory that there was a causal link between tobacco and cancer… and it wasn’t piles and piles of scientific studies that convinced anybody, it was a legal and political battle.

The “Climate Change Debate” is just like that.  There isn’t a climate change “debate” among climatologists.  There isn’t even really a climate change debate among scientists in general (a couple of outliers, none of whom study climate science, does not a debate make), nor is there serious belief in a  “non-anthropocentric cause”.  Not because scientists are out to get rid of technology (I have yet to meet one that didn’t love his or her computer).  Not because they’re out to halt progress (you pretty much need to be on board with the *idea* of progress to become a scientist in the first place).

It’s because the evidence for other theories isn’t there.  There are several major research journals in climate science, and (unsuprisingly) the Caltech library subscribes to electronic versions of them all.  When someone pointed me at Senator Inhofe’s web site, claiming that there was peer-reviewed science that refuted global warming, I went looking for it.  I didn’t find it.  Irritated that I had no actual citations to start with (something I find to be a general media failure, so I could hardly assume that the lack of such was immediate evidence of nefariousness)  I looked again.  I still couldn’t find it.

So I went out on the general Intranet and tried to find any sort of reference to the actual journals these papers were published in, or what their titles were.  I didn’t find that, either, but I did find a whole pile of blog posts by scientists on Inhofe and dismantling the claim into teeny, tiny little shreds.

If you believe that global warming is bunk, you’re very, very likely to be very, very, horribly wrong.  Not guaranteed wrong, of course.  Again, science is not mathematics.  We can’t say that anything is definitively true, because we don’t know for certain what all the axioms of the Universe are.

So what?

We also can’t say that it’s definitely true that if you stick a loaded gun against the side of your noggin and pull the trigger that you’re going to die.  The gun might not fire.  The bullet might be a dud.  The gun could explode in your hand, and just cause a terrible injury, or the bullet might bounce off your skull or by some random roll of the dice not hit anything critical on its way through your grey matter.  Maybe we’re all plugged into the Matrix and the truth is, there is no gun.

I wouldn’t bet on it.  It astonishes me that so many people are not only willing to bet on it, they’re eager to do so.

Germane to my last post, check this out (from Wired, via Bruce’s blog):

Researchers at the University of Utah have found a way to see through walls to detect movement inside a building.

The surveillance technique is called variance-based radio tomographic imaging and works by visualizing variations in radio waves as they travel to nodes in a wireless network. A person moving inside a building will cause the waves to vary in that location, the researchers found, allowing an observer to map their position.

Add a nice little HUD and you could have your own personal radar, tracking all movement inside your evil genius lair.