I recently bought a refurbished router, the same model I’ve used elsewhere in my house with no problems. With THIS one, however, I found myself unable to allow SSH traffic to pass the switch, even with the router in bridge mode. It’s very weird, and probably nothing, but it made me wonder: wouldn’t a great cyber implant strategy be to buy router equipment, replace one or more of the chips with a compromised version that does some nefarious thing (e.g. opens ports to the outside, acts as a node in a DDoS attack, etc.) and then return it to the store for a refund? The store then takes it back, cleans it up and resets it (but doesn’t, of course, notice the new chip soldered in as they don’t have the process to check for that) and resells it as refurbished. If I were a state actor looking to get stuff behind people’s firewalls, refurbished items would be a great attack vector.
Given that pretty much every company that sells routers sources their chips from Asia, wouldn’t it be easier for a state actor to just put the implants in the chips themselves? Well, I don’t think so. That would be too risky and traceable, and eventually production spot checks would likely find something. The process for refurbished products, however, is much less rigorous, and anything found would be impossible to trace back since the product was out of the supply chain for so long.
The journal Science recently published a fascinating article from Alain Cohn et al, which looked at cultural proclivities for civic honesty around the globe. They employed a rather ingenious method: they “lost” wallets all over the world and recorded when the receiver of the lost wallet attempted to return the wallet to its rightful owner. The wallets were fake and included a false ID of a person who appeared to be local to the country in which the wallet was lost, including fake contact info that actually belonged to the researchers. The ingenious element of the research was that instead of leaving the wallet out in the open, the research assistants actually pretended to have found the wallets in our nearby local businesses and turned in the wallet to somebody working in that business, thus enabling them to record interesting ancillary data on the “subject,” such as their age, if they had a computer on their desk, and whether or not the person was local to the country. Clearly, the researchers were hoping to engage in a little bit of data mining to ensure their not insignificant efforts returned some publishable results regardless of the main outcome.
As it turns out, they needn’t have been concerned. The level of civic honesty, as measured by wallet return rates, varied significantly between cultures. In addition, there is an interesting effect where the likelihood of the wallet being returned increased if there was more money in it, an effect that persists across regions and which was evidently not predicted by most economists. I encourage you to read the original article, which is fascinating. On the top end of the civic honesty scale are the Scandinavian and Northern European countries, with rates at around 75%. On the bottom end of the curve is China, with about 14%. In the case of China, all the study did was confirm what anybody who does business there knows, and something that has been well covered by journalists and completely ignored by our politicians: to the Chinese, not cheating is a sign you’re not trying hard enough.
Here’s where things get interesting: in keeping with modern scientific publishing standards, the researchers made their entire dataset available in an online data repository so that others could reproduce their work. There are a lot of interesting conclusions one can make beyond what the authors were willing to point out in their paper, perhaps due to the political implications and the difficulty of doing a proper accounting for all the possible biases. However, unburdened by the constraints of an academic career in the social sciences, I was more than happy to dig into the data to see what it could turn up…
Perhaps the most interesting thing I found is that women appear to be more honest than men. Over the entire world-wide dataset, women returned the wallets about 51% of the time, versus 42% for men. It is tempting to look at individual countries, but the male versus female difference is not statistically significant enough when looking at individual countries, so I chose to only look at the aggregate data. The data is not weighted by country population, so one should take the absolute magnitude of the difference with a bit of skepticism. However, looking at the individual country data it appears a proper accounting for population bias would likely maintain or increase the difference. (Some of the most populous countries had the largest difference between women and men.)
Worldwide, women appear to be statistically significantly more honest than men. Standard error was less than 1% for both cases.
Here is the full dataset of men versus women broken down by country. You can see that the most populous countries are those where women appear to be more honest than men, so fixing the chart above to account for sample bias would likely still find a significant difference.
Women appear to be more honest than men in most cultures, though the individual country results are not usually statistically significant.
Another interesting question to ask of the data is whether or not there is a generational difference in honesty. Surprisingly, the answer turns out to be that there’s not a statistically significant difference:
Age doesn’t appear to be a statistically significant predictor of honesty. Standard error was roughly 1%, so the difference shown is not meaningful.
Looking at the breakdown by country, we see that there are no big differences between the generations, with one exception that I’m not even going to try to explain:
It’s possible that the young are more honest than the old, but it doesn’t appear to be statistically significant except in one country.
One interesting set of issues that always comes up with population studies like this is what, if anything, should we do with this information? It is true that a Swedish woman is about eight times more civically honest, on average, than a Chinese man. That’s interesting, but also pretty dangerous information. Should this inform our immigration policy, where population statistics might actually be valid? Is it better to not even ask these questions given the abuse of the information that might result? Or, is it good to have this information, especially when it flies in the face of our image of ourselves and others? I suspect in the case of the US, most would be surprised to find out that the average US citizen is as honest as the average Russian. We may be surprised by both halves of that statement, and both might be good to think about.
Despite all dictates of logic against it, technical analysis actually works some of the time. I’m not talking about the random luck of any scheme working, but the fact that an uncanny amount of the time, “support” and “resistance” theories seem to be borne out in actual stock data. Assuming I’m not just imagining this, the obvious answer as to why is that it’s a self-fulfilling prophesy. If enough people believe in technical analysis, the patterns inherent to its doctrine will occur because the market is just composed of humans and their beliefs. It’s no more odd than the fact that stocks go up when earnings go up, except that belief at least has some basis in fact. The point is, having a basis in fact is irrelevant to the market.
Also obvious, however, is that when a stock pattern really works, it will draw attention to itself and it will stop working as people try to capitalize on it. Thus, while it may be a self-fulling prophesy that technical analysis will work, the same logic suggests that it should also fail, on average, because the market is mostly a zero-sum game. So, what gives?
It occurred to me that we are missing one vital factor: there are a lot of people who think technical analysis is utter hogwash. Ironically, however, if enough people think technical analysis is bogus, it just might work. It then dawned on me what technical analysis may really be and why it might actually work: a legal form of market collusion.
Technical analysis may work out to be a somewhat effective form of conspiracy among a subset of the market participants. The rules of TA are essentially a language, allowing adherents of the art to manipulate the stock market through mass distributed collusion. I think it is entirely possible that the “rules” of technical analysis are successful because they manipulate the market in a way that gives an advantage to adherents of those rules relative to those who don’t.
The net effect is that a group of people in the market cause prices to oscillate by all “colluding” to buy and sell at certain points. Another group of people then end up making trades at what the other half know to be the “wrong times” in a predictable cycle. The common rules of TA form a system of collusion that doesn’t require any back channel conversation between participants, and thus it is completely legal; the market itself is the channel through which the mass collusion occurs. The general disregard among academics for the possibility that TA could work results in a situation where there are always enough market participants who aren’t in on the game so that the game works. In fact, perhaps the best thing that ever happened for TA was the publication of “A Random Walk Down Wall Street.”
UPDATE: This script is no longer necessary given Apple’s introduction of automatic duplicate detection to iOS. I’ll leave this post here for archival purposes, however.
If you have a huge Photos library on the Mac, you very likely have a bunch of duplicate photos. People have also found that duplicates photos are sometimes created when you upgrade from iPhoto or Aperture to Photos. Unfortunately, if you look at the “free” programs on the App Store, you find that they are generally either scams or teasers for a paid version.
So, I wrote an AppleScript to go through every photo selected in Photos and create a new Album with just the duplicate photos found. (With a large album I recommend selecting one or two years at a time to avoid issues with memories.) The script can be copied to your Photos script folder (if you have the script menu enabled) or you can just run it from with ScriptEditor. I also created a small script to reveal the ID of a selected photo, which can be useful for debugging the library and/or finding an original file within the inscrutable Photos database. Most people (if anybody) will just want the Find Duplicates script. Download them at the link below:
Once the script finds any duplicates, it creates a folder called “Duplicates” and adds a new album with the found duplicates. You should then go through that album and confirm that the duplicates are legitimate. If they are, select which the one duplicate you want to remove (you won’t want to get rid of the entire pair, presumably), right click and select delete. Make sure you don’t delete both in the pair, but if you do it’s not the end of the world; they will stay in the deleted photos album for a month.
Let me know if something doesn’t work right. The Photos AppleScript library doesn’t allow any destructive operations other than deleting albums (which I don’t do in either script) so the worst case should be that if it doesn’t work it just doesn’t find duplicates that it should.
As a couple folks pointed out in the comments, the next step will be to create a script that allows you to “merge” a pair of photos, moving all the album associations and tags onto just one. I haven’t looked into whether this can even be accomplished with the AppleScript hooks available in Photos, so if anybody has an insight I’d appreciate it.
I had originally made plans to move this to a private server, but since I’m working at MIT Lincoln Lab, it looks like I’m going to be able to keep all of my MIT computer access from when I was a graduate student. So, this will stay up indefinitely. I know you’re relieved. I may even start writing posts again…
I ran into what is apparently a not uncommon problem with iPhone video: you start to take a video while phone still thinks it’s in portrait orientation (long axis of the phone vertical) and then the rest of the video is stuck that way, even if you took 99% of it in landscape mode (the way God intended). You thus have a video that the Mac always wants to display sideways. If that didn’t make any sense, the bottom line is I had a video I needed to rotate 90 degrees, and maybe you do, too. While there were plenty of solutions available on the PC, tons of Googling turned up virtually nothing for the Mac, short of finding an old copy of iMovie from five years ago.
Fortunately, I lucked in to a great solution, which doesn’t even require transcoding (with the attendant loss in quality that would result). By the way, this worked for me in getting a video from portrait to landscape, and I suspect it may only work in that situation. (That said, this should be the only valid situation in which this problem occurs, as nobody in their right mind should ever shoot a video in portrait mode, and if you do, I’m certainly not going to be complicit in aiding and abetting that crime against humanity.)
The solution requires a copy of the very nice all-purpose video player, VLC.
Open the video in VLC
It should actually open up in landscape orientation, regardless of the erroneous orientation data in the movie file from the iPhone.
Select “Streaming/Exporting Wizard” from the File menu.
Select “Transcode/Save to file” and click next.
Use “Existing Playlist” and select the file you just opened below, click next.
Leave everything untouched (i.e. both check boxes blank) on the transcode screen and click next.
Choose MPEG4, click next.
Click the button to tell VLC where to put the output file, then click next.
Click finish.
This should be all it takes. The process will be fairly quick, since there’s no transcoding, but its not instantaneous as it does have to move a lot of bits into a new file.
For some reason I decided to take a look at the top 50 most played songs on my iPod. Here they are, listed from 5 to 1.
Rank Title Artist Number of Times Played
5. Many the Miles Sara Bareilles 66
4. Fireflies Owl City 67
3. The Wider Sun Jon Hopkins 81
2. Baby Monkey Parry Gripp 322
1. Oatmeal Parry Gripp 393
One guess as to which two songs are Alex’s favorites?
