Badger Problems?

The title in this recent paper in PLoS One says it all, really: Effectiveness of Biosecurity Measures in Preventing Badger Visits to Farm Buildings. What is a paper on badger control doing in a highly respected science publication?

For one thing, it shows us an important fact about science: Science is a method. It is not about what you study, but about how you study it. Preventing badger incursions is legitimate science when you go about finding out in the proper way.

What is that proper way? There's many descriptions out there, focusing on philosophical or practical aspects. But the gist really is that you find answers to questions without getting fooled — fooled by bad data, fooled by badly posed questions or fooled by your own biases and expectations. It boils down to making sure that your results are real, not an artifact of noisy data or influenced by what you wish were true.

This paper is a well-done bit of science, and it also addresses a real problem. Badgers visit farms, presumably to find stuff to eat, and when they do they can infect the cattle with a version of TBC. Badgers and cows don't come into direct contact very often, so the disease probably spreads indirectly, through badger faeces and urine. Stopping badger visits seems like a good idea.

But farms are big and badgers are small. Stopping them altogether is difficult and expensive. It would be great if you could use simple ways to just keep badgers out of the cattle feed storage, where the risk of spreading infection is greatest. But you also want to know what happens if you do that. Say that you stop badgers from getting into the feed but the badgers simply spend more time where the cows sleep instead, then you may have gained nothing.

 
The authors set up surveillance cameras on a number of farms. For a whole year they simply recorded the number of badger visits to various areas on the farms. Then they picked farms with badger visits (not all of them had a badger problem) and divided them into three groups: One got badger barriers around the cattle feed area; one around the sleeping area and one got both. Spend another year recording (and analysing all those those hours of recording must have been painful for some poor graduate student) then analyze the results.

What did they find? All barriers were effective in reducing badger visits, and barriers around one area reduced, not increased, visits to other areas as well. Barriers around both areas were most effective, but the feed area barriers were almost as effective.

Think of what they did to get to that result. They decided at the outset what question to ask and what would be an acceptable kind of answer. They first just recorded visits so that they'd know the baseline of visits, and they spent a whole year doing so, since the badger frequency likely changes over the seasons. Then they changed things systematically — add barriers to one, other or both areas, covering all possibilities — and recorded for another year.

And they were careful to set precise criteria for the recording and analysis beforehand — what does and does not count as badger visit, what to do with the data if a camera doesn't work part of the night, if a barrier was down, that sort of thing — so that their own expectations wouldn't influence the analysis. They were careful to use statistics to find out how likely the changes they saw were real and not just due to chance.

Whether you're looking for badgers or bosons, it's this painstaking attention to remove errors, uncertainty and biases that makes it science.

While I was otherwise preoccupied…

We have been busy with family issues lately, but the fast-paced, exciting world that is Japanese politics never stops. Very briefly, the Noda cabinet is trying to push through a consumption tax increase; as a result of that as well as a number of other issues, a number of lower house DPJ members have quit the party in protest.

Corey Wallace argues that without a functioning LDP, the DPJ is losing its reason to exist and is in real danger of splitting up. It's difficult for me to argue with that. The DPJ started as the not-LDP party, and has no ideological compass to fall back on now that the LDP is defeated. The greater question is whether a more viable set of parties will ever emerge to take the place of the twin trainwrecks that currently vie to not-really-rule Japan. Right now I am not optimistic.

But this is the New Year holidays. Forget about politics for a while. I am going to spend my time enjoying the company of my wife, with food and drink and lazy strolls in the sunshine. I suggest you do the same¹.

--

#1 With your own wife of course; mine is already occupied. Or your own husband. Or significant other in general. A beloved pet is nice too. Just don't take your goldfish for a walk; they don't appreciate it.

Guile II

I've started learning Scheme, and my system of choice is Guile, a version of Scheme designed to be used as a scripting language for Unix-like operating systems. I'm using the older 1.8 version that is available in Ubuntu right now. The new, much improved 2.0 version will be available in the next Ubuntu version but for now 1.8 is plenty for me to learn the basics.

I have been programming in one way or another for all of my adult life. My experience ranges from embedded systems to very large machines and in a variety of languages. I have never really used Scheme, though, and never tried to work in a largely functional programming language before. The code and all the explanations here is the work of a beginner trying to learn about Scheme programming, and it's bound to contain a lot of bad coding style, suboptimal solutions, misconceptions and outright errors.

So why do I do this? The most effective way to learn something is perhaps to explain it to others. It forces you to make everything explicit, and will expose any doubts or misunderstandings you may have. Now, I don't expect anyone to actually follow these posts in any detail; you, collectively, are the imaginary audience I need for me to to write this.

Guile is Scheme. What does Scheme look like? Here's a bit of scheme code:

;; create a list of values from low to high, inclusive
(define (range low high)
    (if (> low high)
      '()
      (cons low (range (+ low 1) high))))

This defines a function called range that will give you a list of numbers, from low to high:

>(range 1 5)
=> (1 2 3 4 5)

This is a pretty useful function, and not surprisingly a variant (named iota for obscure reasons) is available in a Guile library already, along with a more powerful function called list-tabulate that lets you use any kind of list-generating function.

First, note that there's plenty of parentheses. Lots of them, in fact. Both programs and data use the same notation, and it's quite easy to treat a program as data or the other way around. On the other hand, all those parentheses get quite confusing, so a good editor that shows you where each pair belongs is really helpful if you're going to write Scheme code. I use Vim; other people like Emacs.

Scheme, like LISP, uses prefix notation. That is, first we give the operation, then the data to operate on. So if we want to add two numbers we'd do (+ 2 2), and the expression would be replaced with 4¹. The if-statement comparison shows the same order: if (> low high) would be if (low>high) in many other languages.

Simple statements like numbers or variables can stand by themselves. Any operator or function has to be the first element in a list followed by its parameters. When the function returns the whole thing is replaced by the result. Makes sense, I think, as that's the only way to know what arguments belong to an operator. The range call in the last line, (range (+ low 1) high) has two parameters, where the variable high simply gets replaced with its value, while (+ low 1) is a statement with operator + and two arguments for + to add.


The fundamental way to loop in Scheme is by recursion. We wrap the code we want to loop over in a function². We repeat the body of this function by calling the function again at the end. So we call range with two parameters low and high. If low is smaller, we make a list pair with low as the first element, and the result of calling range with the next value of low and high as parameters. Once they're equal we return back up and build the list on the way. Proper lists end with the empty list value, so we return '() for the final element ("'" is a quote, so the list is not evaluated).

This sounds inefficient perhaps, but it is not. When the recursive call happens at the end this becomes just as fast and efficient as a regular loop. Scheme has other ways to create loops, such as do and while statements, but they are all defined by tail recursion like this.

This is what a non-recursive version could look like, using a while loop:

;; list range function, iterative version
(define (range-iter low high)
  (let ((acc (list high)))
    (while (< low high)
    (set! high (1- high))
    (set! acc (cons high acc)))
    acc))

The let statement creates new local variables. We need an accumulator to store our list, so we start by setting it to a list consisting of the high value. Then, while low is smaller than high, we decrement high, concatenate the new high value to the front of acc, then store that new list in acc again. The final statement is acc which gets substituted with the list it contanins, and becomes the return value of the function.

To me this is perhaps easier to understand, but it's not as elegant as the first version above, and it was more difficult for me to get right.


Now, I said that when the recursive call happens at the end, Scheme can optimize it so it's just as efficient as a regular loop. When the recursive call happens last there is nothing left to do at that level. Scheme doesn't have to keep track of each level, and can return directly to the top at the end. But if you look at the first version, the range is not, in fact, the last statement; cons is. The first version is not properly tail-recursive in other words. Something like

> (range 1 50000)

will fail with a (stack-overflow) error. Scheme has to keep track of each level so it can do the cons at the end. We have to rearrange things so that the cons happens on the way down, not when going back up. We assemble the next part of the list and send it along to the next iteration with an extra parameter. We also define a wrapper function without the extra value just to make it neat — remember, functions are cheap.

;; list range function, properly tail-recursive
(define (inner-range low high acc)
  (if (> low high)
    acc
    (inner-range low (- high 1) (cons high acc))))

(define (range low high)
    (inner-range low high '()))

This works as expected. It turns out though, that the iterative while version is actually slightly but consistently faster than the recursive one for calls up to about two million elements. At that point both versions start slowing down (memory allocation and management starts to become a real problem), but the recursive one slows down more. while is conceptually defined in terms of recursion in the standard, but I guess in practice it's implemented and optimized separately in the interpreter for efficiency.

Note that this is for the ageing 1.8 version of Guile specifically; the newer 2.0 version of Guile or other Scheme implementations may well have faster recursion. And in practice, a 10% difference in speed isn't very important compared to readability and code clarity. If speed really is critical, you're not coding in a dynamic language anyway.

--

#1 We normally use infix notation, where we put the operator between the values: 2 + 2. Why prefix notation? There's a number of reasons, but one is that you're not limited to two values; you can do (+ 1 2 3 4 5 6 7) and get 28 in one go. Also, it makes all operators and functions behave the same, which I guess is sort of important for the kind of person that keeps their canned goods sorted and arranges the family toothbrushes by color and size.

Oh, and is there a postfix notation too? Oh, yes indeed there is, and it's surprisingly useful and easy to work with. I don't care much for prefix notation, but postfix is my favourite way of doing calculations; I guess it mimics the way we do arithmetic in our heads already.

#2 functions are really cheap and easy in Scheme. Defining lots of them at every turn seems to be quite normal and using more but smaller functions seem to be encouraged. It's called "functional programming" for a reason.

Guile

With an end-of-year project sprint, the Japanese language test and family health issues, I been feeling a little drained lately. I'd normally relax and recharge by studying Japanese, by doing photography and by writing on this blog. But I want to get away from studying for a while after the JLPT test; I don't have the uninterrupted blocks of time needed for camera walks and processing the results; and most blogging feels uncomfortably close to work right now, aggravating my stress rather than reducing it.

So, I need a new interest. Something I can do wherever and whenever, and in small snatches of time. Something I know I enjoy. Something like learning a new programming language¹.

A student programming a robot. Good "programming" illustrations are hard to find.

I've enjoyed programming since before high school, I've worked as a programmer and my undergraduate subject was computer science. All you need is a computer. Most languages have lots of good documentation and tutorials online, and you can bring a book for when a laptop is too much. You can read and practice in short increments whenever you have a bit of time. And programming is just like doing puzzles or word games: you give your mind a challenge, occupy your thoughts for a while and emerge refreshed and happy.

But what language? There has been an explosion of new languages, or new interest in old languages, recently. Haskell, Erlang, OCAML, Javascript, Dylan, Scala, Go, Groovy, Kawa, Dart, Clojure… They all promise to be hip, webby, cloudy and concurrent, and they all claim to be the Next Big Thing in computing. Hard to choose. Even harder to choose right. Ideally you'd want to know what a language is not good at, but that kind of comparison is rather hard to find. And the Next Big Thing more often than not turns into a Has Been overnight. Popularity is not a good way to choose.

Better to pick something that teaches fundamentals. One thing many, though not all, of the languages above have in common is that they are functional to some degree. Big, established languages like Python and Ruby also support functional programming. So lets go back to basics, I figure. Let's learn Scheme, and specifically Guile.

I know about functional programming of course. And I use functional constructs in Python and other languages from time to time. But I've never tried to really live a functional life as it were, and I don't have an intuitive sense of how to do it right. By learning and using Scheme I will hopefully not just gain a new language but also enrich my programming abilities in general.


Scheme is a LISP-like language, but very minimalist². On one hand, it makes Scheme an easy language to implement, and it has made it a fairly popular scripting language. On the other hand, the base Scheme specification is so spartan it is very difficult to use. Fortunately, like LISP it has excellent support for extending the language with new syntax, so all implementations are fairly full-featured in practice. Small but extensible is the best of both worlds in a way, but as each implementation does things in different ways, Scheme programs aren't very portable between systems. Even seemingly basic things like looping are actually add-ons to the base language.

Guile is meant to be an embedded scripting language for Linux and other Unix-like systems. That is, it's made to be easily connected to low-level code written in C. You can write the main code in the low-lever language and use Guile as a simple scripting layer. Or you can make Guile the main part, and use C to extend Guile with fast application-specific functions. Or you can skip the low-level code altogether and just use Guile as a development language in its own right.

As it is easy to connect it to lower-level code there are pretty good bindings to many Linux system libraries; there's support for threading and parallelization; and like most Scheme variants it has a very good numeric stack with built in seamless support for exact values (using fractions rather than reals), very large values, complex numbers and more. It is a full-fledged development system, not a toy language.

Guile version 2.0 was released this year with performance improvements, support for compilation, new, better libraries and much more, but for now I will use version 1.8, as it is the current standard version in Ubuntu. The extra features of Guile 2 do not matter while I'm still trying to learn. I thought I'd post about my progress here from time to time.

--

#1 I know I just lost about 90% of you reading this. It's OK; we're all different and so are our interests. Just skip these Guile posts if you're not interested.

#2 Scheme people sometimes brag that the entire Scheme specification has fewer pages than the introduction to the Common LISP standard.

Not Exactly The…

…single best week of our lives, and it's only Tuesday. While we wait for the rest of the week to improve, here's a few recent Osaka shots as a pick-me-up.

Young office worker does a bit of paperwork on the road as it were. Crysta shopping mall, Nagahori.

Yodogawa river between Nakanoshima and Umeda.

Hostess on a second-floor hall in Kitashinchi, Umeda.

JR has a new, huge station in Umeda, with malls, exhibition centers and several public plazas. Beautiful, well-designed place.

Platform entrance, JR Umeda station.

View from JR Umeda station. This, by the way, is my new desktop background.

All pictures were taken with a 1970's Minolta SRT-101, using Kodak's new Portra 400 negative film. It takes more time to process and scan, but other than that (a big "other than", to be sure) I find few reasons to recommend a lower-end DSLR over this set-up.

It's All Text

I dislike Google's new version of Google Reader and I'm not alone. The basic problem with their redesign was that button bars and other things took up a lot of vertical space, leaving less than half of the vertical space for the actual text I want to read. They've since re-redesigned it to reclaim some of that lost space but you still lose about a third of the scarce space. I've since turned to another news reader that — while infuriating in other ways — at least gives me all available vertical space for the text.

Gmail has been redesigned too, and it's just as bad. When you reply to a message, less than 40% of the vertical space is available to, you know, actually write your message. Look at this screenshot:

Gmail, while replying to an email. Note how the actual writing area is just a small window in the lower half of the screen.

So why am I not annoyed? Why am I not abandoning Gmail? Shaking my fist impotently against a cold, uncaring sky? Because It's All Text. This is an add-on to Mozilla that lets you edit any web page input using your own favourite text editor.

Look at that screenshot again, and note the yellow-green circle? There's a small button there that says "Edit". Click on it (or choose It's All Text in the right-click menu) and your editor will open with the current contents of this text input area. Write away, in full screen, in your favourite tool. You save the text and the plugin automagically inserts it right into the web page, as if you had typed it in yourself.

This is wonderful. I think you can use almost any text editor you want as long as it can save as normal text (no word-processor, obviously). I use Gvim, but you can use Gedit, Notepad, Textmate, Emacs or whatever you want. Works on Linux, Windows and OSX (though there's apparently a few extra steps for OSX). It's not just the convenience of editing in a real editor. You can save drafts or copies of what you write, and if you leave the editor open until the text is posted your text will be safe even if the website manages to lose your input. If that happens you just reopen the input page and paste your text into the field - no more rewriting your entire text.

The one place it doesn't work right now is on Google+. A few other web sites manages to lose the "edit" button in the corner, but It's All Text is still available and working through the right-click menu. But on most websites it works just beautifully.

Sweden

At long last, pictures and notes from our vacation in Sweden this September. Still stuck writing for work, so I'll mostly post images with captions and short fragments. Less to write, less to read. Everybody wins!

Stockholm city hall. The Nobel Price award dinner is held here every December.

We flew with Finnair from Osaka to Stockholm via Helsinki. It's the most convenient route by far, and a good choice for any destination in northern Europe. On the way to Helsinki I saw Princess Toyotomi. It's a recent movie set in Osaka, based around the idea that a heir of the Toyotomi clan escaped the sacking of Osaka castle some 400 years ago, and the descendants are still secretly living in Osaka as royalty of a secret "Osaka nation".

It's a charming adventure movie. There's some fun characters and an implausible but creative story line. It sags a bit toward the end though; I guess they didn't know how to tie it all up. The main pleasure for me was the city itself; large parts of it is set right around where we live — at the Karahori market street, Osaka castle, the City hall and so on. Fun movie, so see it if you get the chance. Anyway…

Skandia, a movie house in Stockholm, still with the classical logo.

Vasagatan, Stockholm.

We stayed at a hotel right next to the train station. Very convenient, what with all the bags and stuff we brought. First night we ate at a Belgian place along Vasagatan called Duvel Café. Pretty good food and a huge selection of Belgian beers seems to make the place very popular for younger professionals to go after work. I kept hearing snippets about cloud storage, Android programming and the next new shiny Apple gadget from the people around us. The people at the next table over were enduring a dinner sales pitch about some cross-platform persistent communications library. In a different life, one where I never went to Japan, I might have ended up as one of the regulars here.

I finally got my plankstek. Steak, sauce and mashed potatoes on a wooden board, finished in a hot oven. This restaurant had such huge portions — that, and I'm no longer used to eating so much — that I had to leave almost half of the food.

Sushi in Stockholm. Specifically, sushi in a cheap Thai restaurant next to the station. Hey, it's all Asian, right?

I wanted to try ramen while in Stockholm. The place I wanted to go to was quite expensive; worse, it was open only open for dinner. Ramen is cheap lunch food to me, so we gave up on that and had a nigiri-sushi lunch set at Kungshallen food court instead. Not bad but not great; the rice was a little undercooked and the flavours weren't very exciting. The miso soup was good, though, and they managed to serve it together with the sushi, unlike the sushi place we tried in Paris last year.

Norstedts is one of the largest publishers in Sweden, and one of the oldest. Their head office "palace" on Riddarholmen island is arguably one of the best addresses in Sweden.

We're leaving Stockholm by train. As are these uniformed, supposedly hard-to-see gentlemen, apparently catching up on paperwork while waiting for a connecting train. There is no way to avoid it is there; I bet Genghis Khan would have conquered the rest of the world as well, but after taking Asia he was held up by piles of paperwork that just couldn't wait…

On our way to Borlänge and my parents we stopped by Uppsala to visit friends. The town (an old university city rival to my own Lund) has been blessed with a new concert hall. It's a popular place, apparently, with a good lunch restaurant on street level (and again with huge portions). It's also nicely photogenic. Here you can see the cathedral, the castle and the university library reflected in the top floor corridors.

Borlänge is an industrial town with a large steel mill and a paper mill, located right in the middle of some of the most beautiful woodland area of the country.

One benefit of small towns like this is, nature is close everywhere you go (OK, so it's not always a benefit, exactly - see "TBE"). The Dalälven river goes right through town, and there's ots of small ponds and inlets like this along the banks. This spot along the river is a leisurely ten minutes by foot from the town center.

Another benefit of small, rural towns: plenty of space. This back yard with a barn or storage shed in the background is right in town, a few blocks away from the main road and close to the pond above. You would have to be very wealthy — and very ostentatious — indeed to have a place like this in, say, Osaka city.

It was good to be back in Sweden for a few days. I got to hear my own language for a change, and spend some time in my own culture. But it did feel a little strange for much of the time. I guess culture and language — slang, catchphrases, current events — is beginning to diverge enough from my own remembered experience that it starts to be noticeable and become just a little disorienting. It's like hearing a new version of an old favourite tune that you haven't heard in years. Not worse than you remember, just a little different.

More pictures in the Flickr set.

JLPT

It's the first sunday in december. Yes, boys and girls — it's time for 日本語能力試験 (nihongo nōryoku shiken), or the "Japanese language proficiency test".

I'm taking level 1. Like I did last year. And the year before. And the year before that. And I expect to fail this year too. I do suspect I'll be decently close; If I am, I may try it for real next summer and study for the test over spring.

I don't really need the test right now, but it would be good to have it done and over with. It's kind of like Pokemon — you want to collect all the levels. And there's a proposal to let a passing JLPT grade improve your chances to get a working visa here; if that proposal is enacted the test becomes fairly important.

Wikipedia — Donate!

Wikipedia is, in a way, the most important site I know. For my specific line of research I of course lean heavily on research papers and books. For most anything else, including fields close to mine, Wikipedia is usually my first stop¹ for anything I want to know more about. It is of course unreliable and superficial — and traditional encyclopedias are even more so — but it is a great "map of the land" and launch pad for more reliable, detailed information elsewhere.

There is possibly no other single resource on earth that is as influential and as important, whether your interest is in the advancement of natural science or the definite lineup of characters in One Piece. Wikipedia is run by a non-profit organization and is currently running a donation drive. You could do worse than go over there and donate a few yen, euro, dollars or crowns. After all, you know that you're using Wikipedia far beyond what you pay anyway, right?

--

#1 …and my second, as I click an interesting-sounding link; then my third as I follow another one; then fourth… And suddenly I realize I've spent the last six hours on the site and now learning all about medieval time keeping mechanisms or something. The site is not so much saving me time as much as redirecting it.

Meet Mayor Hashimoto

The Osaka city and prefectural elections have been called. Hashimoto is the new mayor, and Matsui, from Hashimotos Isshin no Kai party, looks to become the new prefectural governor. Hashimoto is on track on his plans to unite Osaka city and Sakai city to the south.

And with about 2/3rds of the voters approving this plan it seems it may actually become reality. He still needs support in Sakai as well, and ultimately approval from the central government; this is where popular support for that plan specifically can make a real difference. He's nothing if not a doer — things will undoubtedly be rather interesting around here the next few years. Good interesting or bad is the question.

Autumn

It's autumn at last. Cool, clear weather; leaves are turning color; Uniqlo is filled with shoppers looking for warm underwear in anticipation of the winter electricity rationing. Not much time to enjoy it this year, though. This time of year always seems to be the busiest as everybody pushes to get things done before the year-end holidays. And this is the last year for this project to there's plenty of extra pressure to get as much definite results as possible out the door in time for project reviews and final reports.

Each year, everybody with a camera is morally obliged to take at least one picture of colorful leaves. This is undoubtedly a leaf. It is arguably quite colorful. I have hereby fulfilled my obligation.

Speaking of autumn: We sometimes drink hot, sweet, spicy wine in Sweden around Christmas. Warm rice wine is moderately popular here in Japan and something I really enjoy. Shōchū or Awamori cut with hot water is another good drink for cold days or chilly evenings. A new variation to me is umeshu and hot water. Very strong, full sweet flavour and it really heats you up. You feel every sip right out to your fingertips; it's like having a portable heater installed in your stomach. Try it.


And on a scientific note, I posted on G+ the other day about a small Youtube film showing just how bad P-values really are for estimating the quality of experimental results. Check it out here. It's well worth seeing.

The Comfort Principle

Here's a neat way to think about things: spend your resources where you spend your time. He calls it "the comfort principle".

So, if you — like me — spend much of your day in front of your computer, then don't be afraid so spend money to get a really good one that will give you a minimal amount of problems and frustrations. I rarely listen to music on the other hand, so it would make little sense for me to get an expensive pair of speakers. And it'd make no sense at all for me to get a worse computer in order to get better speakers. Somebody who always listens to music and uses a computer only occasionally may take the opposite choice.

Of course, you have to look at the larger picture and be sure you're not optimizing the wrong thing. The author takes an office chair as an example: better to get a good, comfortable chair than a cheap one if you're going to sit all day. But what if you chose not to sit all day long instead? A cheap chair and a desk that lets you stand may be better still. And instead of a really good laptop, what I need is perhaps a job that doesn't put me in front of a computer every waking moment?

Kansai airport departure lounge. Airports and airplanes are exciting and memorable, but that's not where you spend most of your time during a trip.

This thinking doesn't just apply to buying stuff. I have a belt. It's a very nice, brown leather belt with a plain metal buckle that I got for my birthday from Ritsuko some years ago. It's sturdy and simple, and it looks good with any clothes I have. I like it very much.

However, the buckle always, inevitably sets off the airport scanner alarms. Whenever I fly I have to remove the belt and put it on the conveyor belt. And as I've been losing weight I really need a belt to hold my pants up. I stand there in front of the X-ray scanner, juggling belt, computer, phone, change, keys and carry-on bag with one hand while clutching at my jeans with the other. I'd drop something on the floor and scramble to pick it up while the people in line behind me start becoming restless. It's just a matter of time before I screw up and literally drop my pants in public.

So why do I use that belt on travel then? It's the comfort principle. I spend a total of, oh, five minutes or so per round trip valiantly trying to avoid a wardrobe malfunction in security. I spend around 3000 minutes of a three-day trip with my belt securely holding my clothes in place, and looking damn good doing so. I'd much rather optimize my wardrobe for the 3000 minutes of my trip than for the five minutes I spend in security.

If you see somebody obviously ill-dressed or badly packed for the security control, remember that they're probably not stupid. They may just know how to prioritize right.

If you want to sell news, at least get it right

I've mentioned the worldwide drop in violence previously, and I've occasionally touched on the role of media in distorting the perception of violence, crime rates and the perception of risk in general.

Here is a fresh new example, from both Asahi Shimbun and Yomiuri Shimbun, based on a white paper from the ministry of justice¹. They report that "40% recidivism rate among youths released from juvenile facilities". Repeat crime rate among all arrested minors have risen to almost 32%, and has been increasing for the past 14 years! In 2004 the total rate was only about 27%; it's risen 5 points, or almost 20% in these eight years! Panic! Mayhem! Blood on the streets! Mass Hysteria!! Cats and Dogs Living Together!!!

Except, well, not.

Take a look at the graph from Asahi (reproduced here, as newspaper links have a tendency to disappear):

See something interesting? The rate of repeat offences — the red graph at the top — have indeed gone up. But look at the dark blue bar at the bottom: the number of repeat offenders have dropped, from around 4 minors in 10000 in early 2000's down to about 2.7 minors last year. That's a drop of 30%! Let me say it again: the number of repeat juvenile offenders have dropped by a third in ten years. This is good news.

And look at the light blue bars, with total juvenile arrests: from around 14 per 10000 in 2000-2002 down to about 8.5 last year. That's a drop in arrested minors of 40%! So, the proportion of arrested repeat offenders have dropped, but the total number of arrests have dropped even more. That's how the repeat offender rate has been rising — not from any worsening trend, but from two good trends, with one improving even faster than the other.

That's nowhere to be found in the Asahi article though. Even with their own graph in plain view, the text discusses only the repeat offences and never mentions that the total rates are down, not up. Yomiuri notes in a single sentence that the total number of offences has been dropping, with no further comment or any attempt to connect it to the rate.

Why the alarmist treatment? One reason can be that a bad news angle sells more papers, and the editors are more interested in their sales numbers than in the accuracy of their reporting. That, or the reporters at both Asahi and Yomiuri are so innumerate that they do not even realize what these figures are telling them.

I've been interested in the Asahi online subscription, where you get the full paper online at another 1000 yen over the subscription we already have. I'd happily pay for factual, balanced, well-researched news, but shoddy pieces like this are putting me off the whole idea.

#1 I've ben browsing their site but I can't seem to locate the white paper itself; it's either not obvious from the title, or it is obvious but my Japanese is insufficient to understand it. This is unfortunate, as Asahi talks about 40% youth repeat offenders while Yomiuri seems to talk about repeat offenders in general, though possibly about juvenile criminals that get arrested again as adults. I can't check this without the original report. Why can't media put links to their source material now that it's so easy to do?

ColorHug

OK, so you're into photography. Like most of us, you download your pictures from your camera, or you scan your negatives, then process them on your computer. Nice. Fun. Convenient.

But frustrating. Why? Because of color. As in, the color you saw in the scene is not what you see on your computer screen, which is not what you see on your other computer screen or what people will see online or what you end up with from your printer.

This is one solution to color problems, of course. I love black and white; and with no color there is no problem with color casts.

Maiko at Shochikuza theatre in Osaka.

There's lots of reasons. Your camera doesn't really capture the "true" color (if you shoot negative film you know just how malleable "true" really is). A printer can't really reproduce all colors your camera can capture or your screen can show; it fudges things outrageously just to give you a vague impression of similarity.

But one frustrating reason is that your screen isn't neutral. Pretty much all screens have a color cast — they're bluish or reddish, or a green tint, or have some odd color shift between lighter and darker colors. And monitors change as they get older. The backlight changes color with age, and the screen pixel colors themselves can change with years of exposure.

So you carefully edit your pictures to look great on your computer. But if your screen is, say, a bit blue, then you will have added extra red to your pictures to compensate for that. They'll look reddish on other peoples screens, and come out red-tinted on your printer. Of course, their screens and your printer have color casts of their own, making things even worse.

You can't do much about other peoples screens. But at least you can do something about your own. Most image-related software today can handle "color profiles". That is, a file that describes how your monitor (or camera, or printer) handles color, and lets the software compensate for it. If you have a good color profile for your particular screen then your software can take it into account. Everything will look "right", that is neutral, as you work with it. It won't fix other peoples screens of course, but at least they get a nice, neutral, well-balanced image without color casts to begin with. It should look better, at least, if not great.

Why can't we just eyeball the color cast ourselves? Because we are really, really bad at evaluating color accurately. Any hint of the right color and we "fill in" the rest ourselves. The picture above is black and white. I added a few areas of solid color tint to it — red, green and blue areas for the signs in the background, muted orange for skin areas, a few splashes on clothes and bags. All colors are solid tints, and just vaguely similar to the real colors. Most of the image is still in black and white, with no trace of color.

Still, at first glance it looks like a natural color image to most people, and some refuse to believe it's largely black and white until you explicitly cover up the colorized parts. Our brains see the hints of color and fills in the rest by itself.

How do you create a color profile? You use a "colorimeter" — a device that measures the color on a screen, or paper — together with software that takes the measurement and generates a color profile from it. There's a few such devices for sale out there like the Pantone Huey or X-Rite ColorMunki. They work well enough. But the software is not open source, so you're dependent on them to support you in the years ahead. If the company goes bust or they decide to discontinue support for newer OS version your expensive device ends up as a paperweight. They also typically work only on recent Macs or Windows machines.

Enter ColorHug. It's a colorimeter, built as open hardware (schematics and the firmware is available for you), and with open source software for Linux. It's a fair bit faster than other systems, and less expensive too. The developer is gearing up for a first production run, and has just announced an advance order program that gets you a unit at a discount in exchange for helping out with reporting bugs and issues.

This is very useful for Mac and Windows users too by the way. The software is for Linux, but the color profile files are the same for every operating system. You get a bootable CD with a Linux system, so you can boot the CD, calibrate your monitor, then use the profile in your own operating system with no problem. And of course, the client software is open source, so somebody is bound to port it to both Windows and Mac if there is enough interest.

I've sent in my preorder already. Interest seems huge, though, so I can only hope I'll actually get one.

Data collection and analysis app, anyone?

Here's a question for people doing data analysis and programming. I'm looking for a tool that I don't know if it exists:

I often find myself collecting data over time; temperature data, my weight, baking results, lots of stuff like that. I want to be able to very quickly, simply, add data on a daily or hourly basis and do my own exploratory analysis and visualisation. Normally I use a spreadsheet, or hack together a small script to deal with the data, but neither is very convenient.

• A spreadsheet lets you enter data as it comes in, but both data entry and analysis is clumsy and rudimentary, and you soon hit the wall in what you can do with it. Try to write a spreadsheet that correlates your data with the day of the week, for instance.


• Octave, R and tools like that are very powerful. But they're not really geared for this kind of simple daily data entry and presentation.They're really about analysing fixed data sets and don't do interactive data collection very well.


• One-off tools in Ruby or Python will do what I want of course, and in practice it takes less effort than doing this kind of interactive thing in Octave and the like. But it feels like I'm reinventing the wheel every single time.

I'm really looking for a tool somewhere between a completely open-ended scripting environment and a restrictive tool like as spreadsheet; Octave or R but geared towards interactive, daily data collection rather than extensive analysis of fixed data sets.

Is there such a thing?

If not, it may be time to start thinking about creating it. A spreadsheet-like, but more task-specific, frontend, with a good way to enter new data and a real language to do your data analysis. Bonus for being able to generate a matching data entry component for Android phones (can't sideload apps on iPhone).

I'm crossposting this to Google+, and you can also reach me through email as well of course.

Porn and Atheism

PZ Myers of Pharyngula asks why somebody can't both do porn and be a spokesperson for atheism? It's not like pornography is illegal, or even seen as wrong by a substantial population after all. I think he's missing a point. It's not about what is permissible, or what is somebody's right. It's all about communication effectiveness.

Leaving aside the two particular things — porn and atheism — in the post above, you can't really do more than one even slightly controversial or disputed thing in public, and still be an effective spokesperson. When you publicly arguing for a particular cause you can potentially reach anybody who is inclined to at least listen to your arguments. If you are also publicly arguing for a different, unrelated cause, you're likely to lose those who are firmly against this cause, even if they're sympathetic to the other one.

Anybody who is firmly set against your position in one field will by implication tend to reject your position in any other. As you publicly declare your opinion on more controversial issues, the circle of people receptive to your arguments in any of those areas will shrink.

If you are an economist, but also publicly a hippie and a drug liberal then many financial workers (who tend to be conservative and dislike people like you) will dismiss any argument you may have for financial reform, no matter how good how solid, your arguments. People who are sympathetic to drug liberalization may well give your economic arguments more weight than they deserve. If you're an arch-conservative fire-and-brimstone religious leader, then your arguments for, say, community schooling in poor areas may well be dismissed by many liberals and non-believers who suspect you're just trying to push your religion onto more people.

If you do porn movies and argue for atheism, you'll lose potential atheists that dislike porn. And you'll lose religious people that could otherwise view porn in a more sympathetic light. Companies love athletes as spokespeople; they typically do not take public stances on anything else, leaving the reach of the company message as wide as possible.

As private people we are free to live as we want, and argue for whatever we want. But if you choose to try to be an effective advocate of a cause then you do need to limit your public engagement in other areas.

Edit: edited the text for clarity. Never blog before morning coffee.

Short Takes

You can track my work habits pretty reliably through this blog. If I am mostly coding or running simulations I write a lot around here. If I'm mostly trying to work on a paper or a presentation this place goes silent. No prices for guessing what I'm currently doing. The silence here gets worse now that I tend to post short notes on Google+ rather than here. I think I would like a way to repost those fragments around here too in some way. As a test, let me post links here to some recent posts of mine. Is it good, unnecessary, or annoying? Please let me know.

• Vim Turns 20. Vim is a text editor, beloved by many programmers and other people that spend most of their time writing text. It's very powerful one you know it, but the learning curve is quite steep. I love it, and it's one of my main tools.


• I get a lot of my daily information through RSS feeds. On my laptop I've used Google Reader for years, but after a recent redesign it's become unusable to me. I'm looking at alternatives such as Feedly, but I have not found anything that really works for me yet.
  
  If anybody has a good suggestion for a news reader, let me know. My main criteria is that I get the feeds in list form (no newspaper-like layout), that syncs with Google Reader, that it is navigable by keyboard and that I be able to go through items feed by feed (or group by group), not just all new items jumbled together in time order.
  


• Japanese curry goes well with couscous. If you get tired of always having curry with rice, this is a good alternative. My Madeleine cookies are improving, but I still don't get the light and fluffy consistency I want. Any tips for fluffier Madeleines are very welcome.
  

Real Pain, Social Pain

We often talk about emotional trouble as painful. We feel hurt by rejection, we smart from hurtful remarks, We get burned by a bad relationship, our hearts ache for company and so on. There's lots of similar expressions in other languages too; bitterness can be expressed as a form of pain in Japanese (苦痛) and your ears will hurt (耳が痛い) from hearing a painful truth. In Swedish, too, rejection and other negative emotions are painful, and experiencing others misfortune can be heart-cutting (hjärtskärande).

Emotional distress as pain is a good, productive metaphor. But — what if it's more than a metaphor? Our experience of pain is a function of our brains after all, just like emotions are. Bodily pain starts with receptors on our skin and elsewhere, but the experience of painfulness definitely happens in the brain itself.

Amputees can experience phantom pain, where the brain is led to believe there's pain in a body part that no longer exists. On the other hand, many pain relievers like morphine or codeine act on the brain pain centers rather than at the source of the pain; I've heard one person describe the effect of a similar drug as "I knew it still hurt a lot; I just no longer cared."

 

Do Pain Relievers Help with Social Pain?

Now, if emotional pain is real pain — if, in other words, "painful" emotions actually use some of the same circuits as physical pain in the brain — then central nervous system pain relievers, such as acetaminophen — commonly known as paracetamol — should work for emotional pain as well. And this is what a group led by Naomi Eisenberger set out to test recently. They published a paper, Acetaminophen reduces social pain: behavioral and neural evidence about this last year.

Unfortunately, the paper is heavily paywalled and I can't get it from the publisher. By current standards of science journalism we'd be going well above and beyond our duty simply by reading the abstract. But fortunately the authors have put up the paper on their own website: you can download the PDF right here¹. And there's another, earlier paper from Eisenberger, Why rejection hurts: a common neural alarm system for physical and social pain², that lays out a lot of the evidence for a common mechanism between physical and social pain.


They recruited two groups of participants — 62 university undergraduates in total. One group took paracetamol twice a day for three weeks, and the other one took a sugar pill, or placebo. The participants rated the amount of social pain they experienced every day. Social pain dropped significantly³ over time among those who took the pain reliever, while the people with the placebo showed no change. The pain reliever seems to lessen the pain of bad social events.

They also did a brain scanning experiment with two smaller groups, 25 people in total. The groups got either paracetamol or a placebo for three weeks. Then they got to play a simple computer ball-tossing game while lying in an fMRI scanner. They thought they played with two other people, but the game was really pre-programmed. The computer "players" gradually ignored the player and refused to toss their ball to them, making them feel rejected and left out.

They found that those who had taken paracetamol for three weeks had much less activity in two brain areas (the anterior insula and anterior cingulate) that we know are involved in the emotional aspect of pain. But there was no difference between the two groups in how painful that ball-game rejection was to them.


So it does seem that long-term doses — note that the effect took a couple of weeks to appear — of some pain relievers really can lessen the pain of social rejection. But the effect is not big, and it doesn't seem consistent. So don't go eat paracetamol on a daily basis to feel socially better — acetaminophen is not good for your liver, and especially so if you also like to drink alcohol.

fMRI scanner. That's me lying there getting ready for a scan. I wasn't ill or anything; I just volunteered as participant in an experiment. It was a fun experience, though difficult to stay awake for the entire experiment. And as a bonus you got confirmation that there's nothing obviously wrong with your brain.

…But There's More!

As it happens, another group led by Tor Wager did a similar experiment just this year (it's Open Access; anyone can read it). They recruited a group of people that had recently been dumped by their partners and stuck them in an fMRI scanner to find out what areas are involved with social rejection. They ran two sets of scans, one to find areas for social pain, and one for physical pain.

To test social pain they showed the volunteers either a picture of their ex-partner and asked them to think about their rejection; or a picture of a friend and asked them to recall a pleasant experience they've had with them. This was repeated multiple times while their brain activity was scanned. This way you can compare the brain activity with and without the bad experience, and the areas that are active only for the bad experience are probably connected to their rejection in some way.

They did the same kind of thing for physical pain: the volunteers either got burned on the arm⁴, or just pleasantly warmed on the same spot. Again, you look for differences between the painful and the non-painful heating in the brain scans. That should show you what brain areas that react specifically to physical pain, rather than to heat or things touching your arm and so on.

When they compared the two sets of differences, they found that the brain areas that deal with the emotional aspects of pain — the "feeling bad about it" — are activated by both emotional and physical pain. That's the same areas that Eisenbergers group found, and it's exactly what we'd expect. But they also found common activity in areas that deal specifically with physical pain. The emotional rejection activates areas that normally only react to bodily injury, in other words. This is surprising, and previous experiments have not found this.


One major reason, Wager's group notes, may be the level of pain. In Eisenbergers fMRI experiment, people played a simple computer game with strangers who weren't being fair to them. Not nice, but not exactly a major life crisis either.

In this experiment, on the other hand, people that have just been dumped get the picture of their traitorous ex-partner shoved into their face, and are asked to please really think through the whole sordid series of events that ended with them being thrown on the curb like yesterday's garbage. It's a whole different world of hurt, and I can imagine it took a bit of explanation to get this approved by the ethics committee.

So it may simply be that social rejection needs to be strong to actually qualify as pain. Our pain centers don't light up for every touch either; they need a minimum level of hurt to react at all. This could explain the puzzling result from Eisenbergers group, where the pain reliever seemed to have an effect for the students that reported daily social pain, but not when students were scanned. We probably encounter much worse social experiences in our daily lives than the computer ball-game they used for their fMRI scan. The pain reliever would lessen the impact of strong, but not weak, social rejection, just like it has an effect on a real skin bruise or cut but doesn't numb us to touch or slight discomfort.

…As This Is Getting Too Long Already…

The takeaway message is, I think, that the difference between our mental experiences and the physical reality is quite blurred in our brains. Paper cut or hurtful word — by the time it reaches the brain it's all just nerve inputs. There is nothing intrinsically more painful about the signals coming from your skin than from your ears. The difference is only in how our brains treat those signals.

Evolution is ultimately pragmatic. If it is useful to treat strong social rejection as physical pain then it will. It doesn't matter if it doesn't make sense from a design point of view, if it makes for a messy, untidy system, or if it will cause unintended side effects and problems for some distant descendant. The brain is full of opportunistic shortcuts, multiple-use mechanisms and exaptations which makes for a very interesting task trying to untangle it all.

--

#1 Are they breaking copyright by making their paper available like this? Maybe, and maybe not. A lot of journals do allow authors to make "draft" versions available; the difference to the published version is usually little more than the addition of magazine logos and page numbers. And depending on the legal residence of the researcher and of the journal, and on the exact wording of the contract, the researchers may retain the right — explicitly or though fair-use provisions — to disseminate their own paper.

On a more practical level, any journal that tries to sue its (unpaid, and frequently paying) contributors for passing out their own work is probably going to find themselves in a bad public relations debacle, with far worse consequences than the possibility of having lost fifty or a hundred dollars in revenue.

#2 If you're not familiar with the research world, you may not know why Dr. Eisenberger is the last author in the current paper, but the first in this earlier one. Very simplified, the first author is typically the one who did most of the actual research. The last author is their supervisor, or research leader or PI (principal investigator). They may have done parts of the actual work, but more likely provided guidance, original ideas, money and other resources.

Dr.Eisenberger, we can infer, probably did this earlier paper as a post-doc in Dr. Lieberman's lab, then managed to secure funding for her own lab, where she apparently continues her line of research but now as a leader of a group of young researchers.

#3 That is the science meaning of "unlikely to be due to chance", not the everyday meaning of "sort-of important". If you think about it, though, the meanings do overlap quite a bit.

#4 There's ways of using low heat to create intense burning sensation without any actual damage. Still, pain is not something you'd use lightly in experiments.

Research PublicationA Modest Proposal

I love doing science. But some things I love doing less than others. Rewriting papers is one of them. Editing and resubmitting a paper is to research what a wisdom tooth extraction is to a long summer weekend; all things considered you'd really rather be doing something else.

It takes a lot of time to write a long-form paper. The text may go through several revisions over the course of months even before the initial submission, and be picked-over several times by all the authors. The total time we spend may easily be a month or two. Extensive edits or a resubmission can double that time. And a lot of this work is almost invisible; we're debating commas, or precise wordings, or the order of arguments for a minor point in the text.

But very few people will actually read your paper in such detail. Most people who see it will just browse; we all "read" — that is, quickly check the summary and figures — a lot of papers, but we focus in detail only on a few. Some estimate that the average number of serious readers of a paper is around 5. And this probably follows a power-law distribution, where a small number of papers get many hundreds or thousand of readers while most papers get almost none. If your paper isn't in a top-tier journal you can probably assume your serious readership is 0-5 people.

We spend a months worth of work or more on tedious polishing. Meanwhile, almost all of our readers will simply skim the abstract, check a summary of results, look through the bibliography and then move on. Only a very few people — and perhaps nobody — will actually want to know about our work in detail.

 

So perhaps we are all spending our time on the wrong thing. I suggest we stop publishing painstakingly polished 20 or 30-page masterpieces. Instead we publish just a 2-3 page text with an abstract, a to-the-point summary of methods and results, and the bibliography. That will satisfy the vast majority of our readers, and will in fact make it easier for them to find what they want.

Then we meet by video-conferencing with those few who want to know all the details. If we save a month of work by not writing the long-form paper, and a one-hour conference takes a total of three hours with preparation and setup, then we could meet with fifty separate groups and still save valuable time. More likely, as we saw above, only a few people would ever want to discuss the details with us, saving us most of that month of project time. And those that want the details will get something better than a paper: they get the undivided attention of the researcher that did the actual work, and get precise answers to their specific questions.

We record the sessions and put them online. In the near future we'll have automatic transcription of each session as well. That will save the details for posterity and will satisfy most people looking for details, so only those with new questions and novel insights will want an in-person discussion. The just-the-facts summary and the accumulated, searchable discussions will hold far more detail, reasoning and justification of the work than any static paper could ever be able to.


Ok, so perhaps the idea isn't perfect. But it sure looks good to me while I'm sitting here editing a paper…

No More Ikea

Ikea has always been a great place when you live abroad. They sell Swedish foods and drinks, candy and chocolates that are hard or impossible to find elsewhere. We go there every few months just to get pickled herring, cheese, flatbread, liquorice and other stuff.

And of course, this being Ikea you never buy just what you set out to do; we also come home with extension cords, drinking glasses, a new lamp… And you get inspired when you walk through the store. We bought our couch in Ikea, as well as Ritsukos new desk. In both cases we saw them when we were just there for some food, and ended up returning a few weeks later to get them specifically.

No more, unfortunately. Ikea has decided that they are better off selling their own-brand things rather than the popular Swedish brands they used to carry. Most specifically Swedish things are gone: the cheeses, the pickled herring, the bread, the liquorice, Kexchocklad and Dumle. What is left is generic-brand stuff, often not even made in Sweden, and indistinguishable from what we can pick up easier and cheaper in the supermarket at home.

So, no reason for us to visit Ikea now. And as we'll no longer browse in the stores, I doubt we'll be buying any much furniture or home goods from there anymore either. We have plenty of other furniture stores around here after all, with any conceivable style and any price level we'd ever want. From Japanese blogs it seems we're not the only ones; the reaction has been surprisingly negative even among people with no emotional connection to these particular goods.

Seems like a pretty dumb misstep on the part of Ikea to me. I may be wrong of course; the greater profit on own-brand generic foods may more than make up for the small loss of business that results. I don't think so, though. The connection to Sweden is an important part of their brand, and diluting it even further — they haven't been a Swedish company for many years — risks damaging themselves far beyond the loss of a few expats crying over their lost mustard herring.


Now the sole remaining question is where to buy or order pickled herring in Osaka. Anybody know?

Theremin!

I was going to post a different thing early this week but it just sort of fell through. Instead, let us celebrate the very apex in human musical history with this 1980's-inspired original composition for synthesizer and Theremin. The sunglasses really bring this whole video together, I think, and he gets extra credit for the perfectly themed video effects.

Short Takes

No real time for a longer post this week. So, instead, a small collection of notes:

• Google has gotten a lot of criticism for it's real names-only policy. Now it looks as if they may be preparing to back down on it. No word on exactly how and when, but my guess is that this will coincide with the appearance of company and brand-name accounts; they need a public name separate from whoever is actually managing the account so the back-end mechanism would be just about the same. I also guess that they'll still require a real name for registration and only your publicly visible name will be pseudonymous. That'd be good enough for most cases.


• And speaking of Google+, Chris Willson has finally appeared there. He's a travel photographer based in Okinawa, and has been one of the inspirations for my own photography.


• Not speaking about Chris at all, this is a letter from a psychopath to an author who recently wrote a book on psychopathy. It's a good, insightful read. Psychopaths are usually depicted more or less as monsters in news and in fiction. This letter serves to remind us that they — like anyone with a psychological disorder ­— aren't monsters, but just people. Nothing more and nothing less.


• I've installed the new version of Ubuntu on my laptop. It's much as I expected: mostly smooth and working well, but as always with a new version there's a few issues I've had to work around. But I have to say that by now there really are no necessary workarounds in Ubuntu any more. All things I've dealt with are about how I expect things to work after using Linux for many years; a newcomer would no doubt find the current system to be just fine. Recommended.

Network Programming

Network socket programming is fun and challenging, with lots of interesting sub-problems and many chances to learn new things about your systems, your libraries and your compiler.

Or, it is a stress-filled exercise in screaming frustration, a tiny slice of hell filled with deadlocks, race conditions, incomprehensible error messages and intermittent bugs that disappear the moment you try to debug them.

Which is it? That seems to depend entirely on the proximity of your deadline.

Retractions, Corrections and You

If you're involved in academic publishing then this piece in Nature (open to all, as far as I can tell) about the system of paper retractions is a good read. Go ahead and take a look; I'll wait here.

Part of the problem with retractions is, I believe, that there's no middle ground. The article mentions several times that if a paper is retracted people will assume some kind of research fraud. A retraction is stigmatising and people are very reluctant to retract unless forced to as a result. It doesn't help that you lose a publication that took months, perhaps years, to put together.

Perhaps fraud should be the only reason to retract a paper. If the paper is invalid due to honest mistakes then it may be better to let the paper stand, but with a correction and addendum that makes clear what results are invalid, why they are wrong, and what the correct results are. Mistakes — especially mistakes that are serious enough to invalidate a paper, yet subtle enough not to get caught before publication — are an important source of knowledge in itself.

 
If people are reluctant to do corrections, then why not make a substantial, informative correction and reanalysis count as a publication on its own? That ought to give people a bit of incentive to do a good job with it.


The other big problem is that corrections and retractions aren't widely announced, and the information doesn't really trickle down to people using the paper. The article only touches on it, but once you've downloaded and read a paper you're very unlikely to revisit the original site again. Why would you, after all? And with the flood of new papers showing up every day you can easily miss a correction or retraction notice in the deluge. Retracted or corrected papers end up being used for many years after they should have been dumped. I don't have a good idea for how to fix that.

But I don't think the resulting problems are all that dire. Your typical paper has two kinds of references, really: the main sources, and a bunch of supplementary ones. Your main sources are really significant. They're the ones you're really building your work on and without them your paper falls apart. The supplementary sources are more about dotting i's and crossing t's; you're showing that you've read the literature and use them as support for minor points in your work. If one of them happens to be wrong¹ it won't actually have much of an impact on your paper or your work.

And of course you're lot better informed about events surrounding your core references. You seek out and read other papers that reference the same work; you follow the same journals they appeared in; and you keep an eye open for more publications from the same group. If one of your core references are retracted (or simply shown to be incorrect or incomplete) you're quite likely to find out.

This is a general principle, I think: if an important paper is retracted it is a serious matter, but many people will quickly find out and knowledge of it will rapidly spread. If a paper is retracted and nobody notices, on the other hand, then it wasn't much of a paper to begin with and the damage is minimal.

--

#1 It's worth noting that it's common to find several papers about the same exact thing that all disagree with one another. We deal with, and make use of, potentially incorrect papers on a daily basis and have always done so. We know our sources are not some infallible truth, and the sky doesn't fall when a paper or two turns out to be incorrect.

Dennis M. Ritchie

Dennis M. Ritchie passed away yesterday. He was a pioneer of modern computing; one of the fathers of Unix and the developer of the C programming language. From these two foundations has grown most of the software tools and technologies that we all depend on every single day.

He also co-wrote "The C Programming Language". The book, like the language itself, is masterful in its brevity and clarity. It aims to describe the language, but managed to teach me an enormous amount about programming in general along the way, and in fewer pages than the introductory chapters of some programming books. It is one of my all-time favourite books and I still leaf through it from time to time.

Spam

Thought I'd make a quick note here about spam again, as one just managed to get past the normally quite good filters here.

Spam, as far as this blog is concerned, is not just sales pitches or link farm posts. It's basically any comment that tries to hijack the subject to spout whatever the commenter wants to talk about. One and the same comment could conceivably be spam when posted on one post, and perfectly fine when posted on another.

Those kinds of comments is just like that guy at every conference; the one that wants to ask the speaker a question only to spend five minutes droning on about some subject that has nothing whatsoever to do with the presentation. Meanwhile the entire room silently sits there, hoping he'll have heart attack, stroke or an inopportune bowel movement — anything to make him shut up already.

Ozawa and Juholt

As you may know, Ichiro Ozawa, general political heavyweight in Japan, is now on trial for his office having accepted some 400 million yen in illegal campaign contributions, after his secretaries have been found guilty of the same. Opinions seem sharply divided if there was criminal intent; whether 400 million is a lot or not; and whether Ozawa knew about it or should shoulder any blame if he didn't.

If we want to get a sense of what is acceptable, it would certainly help to compare with other places. We're (not so) fortuitously right now getting a point of comparison in Swedish politics. Håkan Julholt is the recently elected leader of the largest opposition party. He has now been found to have received too much reimbursement for his working apartment.

A bit of background: House members in the Swedish parliament have a right to an allowance for a place to live in Stockholm if they don't live there already. The (quite reasonable) idea is that they represent their area in the country and shouldn't have to move away permanently to work as House members. On the other hand they do need to spend a substantial part of the year in the capital, which is an expensive place to live. So you get reimbursed for your share of any accommodation in Stockholm — the operative point here being "your share". Most members do not use this; they stay in accommodations provided by the House.

What Juholt has done for years is claim the entire rent on his apartment, while living there with his partner (they're not married, and I don't think the rules would be any different if they were). He should have been claiming only half. The total disclosed amount he has received improperly is 160 000 crowns, about 1,7 million yen. At this point there are people claiming he was informed this was improper, while he and his people say he was not. There is no publicly disclosed evidence either way.

So, 160k crowns — 1.7 million yen — in allowance he should not have received, knowingly or by mistake. As a result, the police and prosecutors have started a criminal investigation. Every newspaper (including those loyal to the opposition party) are combing through years of old receipts and documents to find more. Some party members are already publicly calling for him to step down. Everybody seems to agree that if he hasn't disclosed the total amount or lied in any other way; if there is credible evidence he was warned about this and went ahead anyway; or if the police investigation leads to charges being filed; then he is most certainly gone as party leader and most likely gone from national politics altogether. Even if nothing more surfaces it's an open question whether he can stay on and be an effective leader of his already wounded party.

Politics is a game of trust. Whether you actually do something wrong matters far less than the perception that you did. I'd give Juholt a 50% chance of political life after this, and that's given that nothing more happens. Similarly, whether Ozawa has done anything illegal or not is up to the courts to decide. But Unfairly or not he is already being judged by the media and the public, and ultimately that's what determines his fate. If you're playing a game of trust it doesn't matter if you're technically legal. The perception of you as upright and honest is the only thing that matters.

Canon Autoboy II

Canon Autoboy II. This one has a recording back that optionally imprints the date and time on the picture.

Something interesting happens when word gets around that you still shoot film. Friends and relatives that clean house show up with old film cameras they would otherwise throw away to ask if you'd perhaps be interested in it instead. And of course I am. By now I now find myself with dozens of old, fun toys to play with.

The Canon Autoboy II - also named "(New) Sure Shot" in the US and "AF35M II" in Europe - is an inexpensive point and shoot camera from the early 80s. Like so many products from that time it sports an angular, futuristic design that would have been quite at home in an episode of Space 1999. It has a pleasantly wide 38mm f/2.8 lens. Not a speed demon, but not bad.

Automation was all the rage by this time, and the Autoboy is fully, almost aggressively automatic. Stick the film in and it will load the film on its own, and set the right ISO based on the marks on the film canister. When you turn it on the lens cover opens automatically (and noisily), and it will set both exposure and distance as you take the picture, with no feedback needed (or wanted) from you.

Stairwell. Daimaru department store in Shinsaibashi, Osaka. The building is wonderful, but while they seem to accept shots like this — especially from someone that seems to be a tourist — photography is generally prohibited. I wish you'd be explicitly allowed to; perhaps they could have a photographers evening some night after closing time?

I don't have a manual for the camera (can't find one online), but it seems you can't control it manually at all. So, no pushing or pulling the film; no exposure compensation; no scale focusing. You can apparently prefocus it by pressing the shutter halfway, but that doesn't work very well as you have no indication where you've focused in the first place. This is not an instrument for precise control. Point and shoot and there's your picture, with minimal fuss but plenty of buzz.

Buzz? This is a noisy camera. It's an inexpensive design and probably uses geared motors throughout. The lens cover opens with a loud whack, the lens whirrs and wheezes as it focuses, the film advance almost screeches, and the shutter has a hard plastic-and-metal burst of rattles that's as noticeable as the mirror and shutter as my Pentax 67. This is not a camera for candid photography.

Hipster, Osaka style. Horie, Osaka.

With its plastic body and noisy mechanism it sounds and feels cheap. I've seen used units — in worse condition than mine but fully functional — in shops for as little as 5000 yen. That's actually a little unfair, I think. The body seems quite solid, and while the camera is noisy the mechanics seem reliable enough.

What about the results? Good. Really good. The images I took (using Ilford Delta 400) have plenty of contrast and good detail, and overall distortion and corner performance seem fine to me. The autofocus and autoexposure systems do a good job; the lens and autoexposure even manage heavily backlit subjects without getting excessive flare and underexposure.

A student works on a Roomba-Kinect frankenrobot. NAIST, Ikoma. Heavy backlighting from the doors in the background, and yet exposure is good enough to make a decent shot. The lens also manages fine without excessive veiling glare. Many cameras would do much worse in a situation like this.

The camera takes normal 1.5v penlight batteries and they seem to last forever, so you'll never be stranded from being out of power. It seems decently solid, better than my first impressions, and the body is sleek with few protrusions. And with the current basement-low prices for a used unit this could be an excellent bag-camera — a camera you load and stick in your bag just in case you ever suddenly need a film camera. It can ride along for months, none the worse for wear. And if something does happen, if it gets drenched in a rainfall or something, well, it's just a few thousand yen for a new one. If you're looking for an inexpensive way to start with film you could do much worse than this.

Down escalator in the new JR Umeda station, Osaka. Fireworks festival the same evening so lots of people around wearing traditional summer dress. I'm a bit envious; summer yukata (or samue) are cool and easy to move in when it's hot, but it looks vaguely ridiculous on a westerner.