Set theory using the shell

What if you have two files with unique lines and you want to figure out what lines that only exist in one file or what lines the two files have in common? Maybe the first things that come to mind is to write a program in say Java or Perl, but the standard shell tools can be just as efficient and no coding necessary.

To find the lines that are in both files (union):

sort file1 file2 | uniq > union.txt

To find the lines that are in both files (intersection):

sort file1 file2 | uniq -d > intersection.txt

To find the lines that are in file1 but not file2:

sort file1 file2 file2 | uniq -u > only_in_file1.txt

uniq -d prints only duplicates and uniq -u prints only unique lines, while sort sorts it's input file(s). In my experience this is at least as fast as a program written in Python or Java with input files consisting of millions of lines.

Well done, please

(Don't) read this before you eat your next piece of meat:
What everyone ought to know about parasites

Only in Japan

Dog owners 'fleeced' in poodle scam

07:10: Restate my assumptions ...

# Mathematics is the language of nature.
# Everything around us can be represented and understood through numbers.
# If you graph the numbers of any system, patterns emerge.

In the latest issue of the Economist there is an article about MSI, the company that owns the Weather Channel. They also sell weather reports to weather dependent companies like airlines. Their latest business idea is to sell a forecast of the forecast released every day by the American met office for 90.000$ per year released one hour before the met office forecast. Observe that this is not a forecast of the weather but rather a prediction of how the met's office simulation of the weather works. Who would buy this kind of forecast? Not the weather dependent industries, as they are interested in the weather forecast and not the forecast of the weather forecast. The answer is investors that trade in weather dependent commodities like fuel. When more and more investors subscribe to the forecast of the forecast its value to investors becomes smaller and smaller. Thus, the next step would be the forecast of the forecast of the forecast that one hour earlier predicts the forecast of the forecast. I wonder how well this one predicts the real weather?

Turkey icecream

Here's the turkey icecream I mentioned before.

More than a feeling

Until yesterday I didn't know that Brad Delp, the singer of legendary 70s and 80s mainstream band Boston, committed suicide about a month ago. What's interesting is that his suicide note that he had attached to his shirt read "J'ai une âme solitaire. I am a lonely soul." Dedicated fans of Twin Peaks remember that this is exactly what Harold Smith, Laura Palmer's secret friend, also did. And, to make the plot thicken, Harold Smith came from Boston...

Wow

Where should I begin? So much has happened since I last blogged so I'll leave it all for a later post. Instead I would like to draw your attention to the "Least Metal moment", Vince Neal performing the Chicken Dance!