There will be no significant update this week as I devote my free time instead to consumption of "Capital in the 21st Century". Expect a follow up review next week.
Until then stay safe and rationale.
Monday, April 28, 2014
Sunday, April 20, 2014
Teaching Value
Public education in America is a tricky subject. Nearly everyone with children would agree that education should be a high priority for tax expenditures. However, a large portion of the population will never utilize the public education system while they are a tax paying citizen (either due to private options or lack of children). Additionally, due to widespread unionization teacher's hold a large amount of political power in the United States. These factors have combined to create a situation where the general feeling is teachers should be better, cost no more and be safe from lay-offs.
Unfortunately, as most reasonable people will recognize, increasing teacher quality without removing the worst teachers or increasing wages is challenging. What is needed is evidence of the benefits better teachers provide. Policies could then be enacted based on that evidence to either increase wages or remove the worst teachers. (Please note that removing the worst teachers does not equate to firing them. Teachers could be incentivized to retire early, or change positions for example.)
Luckily a recent study has provided exactly this evidence. This study by Chetty et al. tracked how teacher quality (as measured by improvement in test scores) translated into a variety of outcomes later in life. Better teachers resulted almost universally into better outcomes for students. Quantitatively, the difference between the bottom 5% of teachers and an average teacher was $250,000 in lifetime wages spread over the classroom annually. In other words, replacing a poor teacher of 25 students with an average teacher of the same 25 students will result in each student earning an additional $10,000 over the course of their lifetime (Mean present value).
Additional quantitative measures for a one standard deviation in teacher value added include:
Unfortunately, as most reasonable people will recognize, increasing teacher quality without removing the worst teachers or increasing wages is challenging. What is needed is evidence of the benefits better teachers provide. Policies could then be enacted based on that evidence to either increase wages or remove the worst teachers. (Please note that removing the worst teachers does not equate to firing them. Teachers could be incentivized to retire early, or change positions for example.)
Luckily a recent study has provided exactly this evidence. This study by Chetty et al. tracked how teacher quality (as measured by improvement in test scores) translated into a variety of outcomes later in life. Better teachers resulted almost universally into better outcomes for students. Quantitatively, the difference between the bottom 5% of teachers and an average teacher was $250,000 in lifetime wages spread over the classroom annually. In other words, replacing a poor teacher of 25 students with an average teacher of the same 25 students will result in each student earning an additional $10,000 over the course of their lifetime (Mean present value).
Additional quantitative measures for a one standard deviation in teacher value added include:
- .82% increased chance to attend college. This is relative to a 37% chance overall.
- 1.3% increased annual earnings at age 28.
- Estimated increase of $39,000 in lifetime earnings per student.(7,000 NPV at age 12).
- Reduced likelihood of teen pregnancy.
- Increased retirement plan savings.
All of these factors are indisputable sources of tax revenue or savings which support policies geared towards improving teacher's value added. Let's set aside any cost savings associated with welfare program utilization rates and examine only the increased tax revenue. If a students increased earnings have a net present value of $7000, the federal government will get (being extremely conservative) at least $700 of that. There are approximately fifty million public school students in the United States (source) and a teacher generally instructs twenty-four students at a time. Given these numbers, if teacher value added can be increased by one standard deviation for a cost below eight hundred and fourty billion dollars it would be foolish not to make the expenditure. This would be approximately a 72% increase over existing total state education expenditures.
If there is one flaw in this study it might be this. Education may be somewhat of a zero-sum game. In other words, Jack may get the better job because he had the better teacher. But that may simply steal the job from Becky. Thus Jack will certainly benefit, but overall tax revenue may not grow. What is needed is a further study that links this data to productivity gains and economic growth. This perspective makes it difficult to enact policies from a federal standpoint. However, at the more local (and state) level it makes the data all the more compelling. Local schools simply need to out compete their neighbors for the best teachers to reap significant advantages. Some benefits will be lost to movement out of the locality. However, the cost to attract better teachers will be significantly less than the cost to improve teachers nationally. Localities only have to offer slightly higher wages than neighbors. At the national level wages would have to rise significantly to attract more of the working population to the teaching profession.
While overall national benefits are still a bit unclear, there are some certainties. Improving teachers significantly improves the future outcomes for students. Test results are an important metric (although change in test results is better) to judge teachers. And there are significant gains to be had by pushing for better teachers. The only question that remains is who is going to start reaping the rewards first.
That's it for this week. Until next time stay safe and rationale.
Friday, April 11, 2014
Golden Balls
There have been some bizarre game shows over the years. "Find the Chair" is exactly as entertaining as it sounds, while "Oh Sit!" is a great pun title for televised musical chairs. But for an economist there's a select few shows which rise above the rest. "The Price is Right", "Deal or no Deal", and "Golden Balls".
American audiences are familiar with the first two shows, but the third is a little known British game show that might as well be taught in game theory courses.
Golden Balls is a simple show of gradual team reduction. A team begins with four players with one player eliminated each round until two remain. In the first round twelve balls are drawn from a pool of one hundred. Each ball has an assigned value between ten and seventy-five thousand British pounds. Four Killer balls are added to the twelve drawn balls and each player randomly receives four balls.
Players then place the balls they receive in two rows. The front row of two balls is visible to all players. The back row is visible only to the owner of the balls. Each player then attempts to convince the others that his or her hidden balls are the most valuable (thus making them important to maximizing later winnings). Players then vote to eliminate one of the team members. Round two is played in a similar fashion with each player receiving five balls total.
When the game is narrowed to two players things really get interesting. Each player takes turns randomly picking one ball from the pool of remaining balls to add to the jackpot and one ball to eliminate from the pool. If a Killer ball is selected to be added to the jackpot then the jackpot is divided by ten. This process is repeated five times in order to establish the size of the jackpot. I've glossed over a few minor details in the interest of brevity but the bottom line is high value balls are good, Killer balls are bad, and the game ends with two contestants going head to head.
Once the jackpot is established each contestant receives one final pair of golden balls. One ball says "split" and the other says "steal". The contestants converse for a time (attempting to convince the other of their trustworthiness) and then choose a ball. Both players balls are revealed and one of three outcomes plays out.
If both contestants select split then everyone wins. The contestants split the jackpot evenly. If one contestant chooses split and the other chooses steal than the stealer gets the entire jackpot. If both contestants select steal than no one receives anything.
It's possible you may recognize this as a modification of the classic prisoner's dilemma. Cooperation benefits everyone, but defection benefits one person more than cooperation would as long as the other doesn't defect as well. Of course if both defect the outcome is unfortunate for everyone.
Interestingly, most shows end with at least one player choosing to steal. It's easy to attribute this selfish play to simple greed. However, interviews conducted with participants seem to indicate that isn't the case. Most players claim that they choose to steal not because they wanted the money all to themselves, but rather because they feared being made a fool of by the opposing player. The fact that fear of betrayal or public embarrassment seemingly supersedes desire for considerable monetary gain is an interesting motivational indicator. Of course, it's also quite possible that this is a post hoc rationalization by players to justify themselves as more than simply greedy.
One player in particular stands out as having beaten the Golden Balls system. Nick Corrigan found himself in the final two showdown with his teammate. When the time came to convince the other that he would split Nick took another approach. He insisted that when the time came, he would select the "steal" ball. Nick claimed that he was going to steal and after the show he would split the money with the opposing player.
At first everyone was shocked. The players argued at length. His opposition implored him, "Why not just both choose split and share the money?" But Nick was resolute, he would select the steal ball.
It seems like a ridiculous plan. After all, nothing compels Nick to split the money after the show. But look at it from his opponents perspective. If Nick is going to choose steal what choices are there? Go along with him and choose split, hoping he'll make good on his promise. Or choose steal and certainly get nothing. There is no fear of betrayal, no public humiliation. Everyone knows what Nick is going to do. The choice for his opponent is simply to hope for something or guarantee nothing.
Eventually, after a long and heated debate Nick's opponent gave in and agreed to select the split ball. True to his word when the balls were revealed it showed "Split". Nick however was not quite so honest. Not one, but two balls read "Split". Both players win and everyone walks away happy. Interestingly, in the post game interview Nick's opponent revealed his planned strategy. He said he'd planned to steal 100%.
That's all for this week. Until next time stay safe and rationale.
American audiences are familiar with the first two shows, but the third is a little known British game show that might as well be taught in game theory courses.
Golden Balls is a simple show of gradual team reduction. A team begins with four players with one player eliminated each round until two remain. In the first round twelve balls are drawn from a pool of one hundred. Each ball has an assigned value between ten and seventy-five thousand British pounds. Four Killer balls are added to the twelve drawn balls and each player randomly receives four balls.
Players then place the balls they receive in two rows. The front row of two balls is visible to all players. The back row is visible only to the owner of the balls. Each player then attempts to convince the others that his or her hidden balls are the most valuable (thus making them important to maximizing later winnings). Players then vote to eliminate one of the team members. Round two is played in a similar fashion with each player receiving five balls total.
When the game is narrowed to two players things really get interesting. Each player takes turns randomly picking one ball from the pool of remaining balls to add to the jackpot and one ball to eliminate from the pool. If a Killer ball is selected to be added to the jackpot then the jackpot is divided by ten. This process is repeated five times in order to establish the size of the jackpot. I've glossed over a few minor details in the interest of brevity but the bottom line is high value balls are good, Killer balls are bad, and the game ends with two contestants going head to head.
Once the jackpot is established each contestant receives one final pair of golden balls. One ball says "split" and the other says "steal". The contestants converse for a time (attempting to convince the other of their trustworthiness) and then choose a ball. Both players balls are revealed and one of three outcomes plays out.
If both contestants select split then everyone wins. The contestants split the jackpot evenly. If one contestant chooses split and the other chooses steal than the stealer gets the entire jackpot. If both contestants select steal than no one receives anything.
It's possible you may recognize this as a modification of the classic prisoner's dilemma. Cooperation benefits everyone, but defection benefits one person more than cooperation would as long as the other doesn't defect as well. Of course if both defect the outcome is unfortunate for everyone.
Interestingly, most shows end with at least one player choosing to steal. It's easy to attribute this selfish play to simple greed. However, interviews conducted with participants seem to indicate that isn't the case. Most players claim that they choose to steal not because they wanted the money all to themselves, but rather because they feared being made a fool of by the opposing player. The fact that fear of betrayal or public embarrassment seemingly supersedes desire for considerable monetary gain is an interesting motivational indicator. Of course, it's also quite possible that this is a post hoc rationalization by players to justify themselves as more than simply greedy.
One player in particular stands out as having beaten the Golden Balls system. Nick Corrigan found himself in the final two showdown with his teammate. When the time came to convince the other that he would split Nick took another approach. He insisted that when the time came, he would select the "steal" ball. Nick claimed that he was going to steal and after the show he would split the money with the opposing player.
At first everyone was shocked. The players argued at length. His opposition implored him, "Why not just both choose split and share the money?" But Nick was resolute, he would select the steal ball.
It seems like a ridiculous plan. After all, nothing compels Nick to split the money after the show. But look at it from his opponents perspective. If Nick is going to choose steal what choices are there? Go along with him and choose split, hoping he'll make good on his promise. Or choose steal and certainly get nothing. There is no fear of betrayal, no public humiliation. Everyone knows what Nick is going to do. The choice for his opponent is simply to hope for something or guarantee nothing.
Eventually, after a long and heated debate Nick's opponent gave in and agreed to select the split ball. True to his word when the balls were revealed it showed "Split". Nick however was not quite so honest. Not one, but two balls read "Split". Both players win and everyone walks away happy. Interestingly, in the post game interview Nick's opponent revealed his planned strategy. He said he'd planned to steal 100%.
That's all for this week. Until next time stay safe and rationale.
Tuesday, April 1, 2014
Flashy Magically Rendered Images (fMRI)
All credit to Pete Etchells for the following article.
Of course April Fool's. Hopefully everyone finds it as amusing as I did. Until next time stay safe and rationale.
A new study has raised new questions about how MRI scanners work in the quest to understand the brain. The research, led by Professor Brian Trecox and a team of international researchers, used a brand new technique to assess fluctuations in the performance of brain scanners as they were being used during a series of basic experiments. The results are due to appear in the Journal of Knowledge in Neuroscience: Generallater today.“Most people think that we know a lot about how MRI scanners actually work. The truth is, we don’t,” says Trecox. “We’ve even been misleading the public about the name – we made up functional Magnetic Resonance Imaging in 1983 because it sounded scientific and technical. fMRI really stands for flashy, Magically Rendered Images. So we thought: why not put an MRI scanner in an MRI scanner, and figure out what’s going on inside?” To do this, Trecox and his team built a giant imaging machine – thought to be the world’s largest – using funds from a Kickstarter campaign and a local bake sale. They then took a series of scans of standard-sized MRI scanners while they were repeatedly switched on and off, in one of the largest and most robust neuroscience studies of its type.“We tested six different MRI scanners,” says Eric Salmon, a PhD student involved in the project. “We found activation in an area called insular cortex in four of the six machines when they were switched on,” he added. In humans, the insular cortex has previously been implicated in a wide range of functions, including consciousness and self-awareness. According to Trecox and his team, activation in this area has never been found in imaging machines before. While Salmon acknowledged that the results should be treated with caution – research assistants were found asleep in at least two of the machines – the results nevertheless provide a potentially huge step in our understanding of the tools we use to research the brain.However, some researchers are skeptical of the findings. Professor Stephen Magenter, Professor of Image Processing at Yate University, UK, is a vocal critic of the statistical analyses that Trecox used. “They just used felt tip pens to highlight and extend the areas they were interested in,” he alleges, adding that he would never colour outside the lines. In response to these claims, Salmon says that this study was one of the most advanced of its kind. “All of our analyses were digital,” he notes. “We used MS paint wherever possible.”The findings raise interesting questions about how fMRI techniques should be used from now on. “If there’s a possibility that MRI machines are showing some sort of rudimentary self-awareness, then we really need to explore this further,” says Trecox. He adds: “One way to do this is to look at what’s happening in our giant scanner, and for that, we’re going to need a bigger machine.”
Of course April Fool's. Hopefully everyone finds it as amusing as I did. Until next time stay safe and rationale.
Thursday, March 20, 2014
Unwinding the Auto Dealer
Half a percent. It's not an exceptionally large portion. Yet it's a significant amount of market share for a company as young as Tesla Motors. It may not seem like much, but it's got traditional auto dealerships worried.
Tesla is reimagining nearly every aspect of the automotive. Everything from manufacturing to sales has been reconsidered from the ground up. Up until now Tesla's innovations have been primarily restricted to the reasonably wealthy. However, it's not difficult to imagine many of their innovations propagating to the majority of automotive sales.
To most people Tesla's largest innovation is likely the manufacture of completely electric, long range, luxury cars. However, the less obvious disruption of their sales philosophy is equally game changing. Tesla sells their cars direct to consumers, rather than through a franchise like most auto makers.
Critics claim that the direct to consumer model stifles competition as Tesla can charge whatever prices they wish. It's difficult to view this idea as anything more than ridiculous. Granted, Tesla monopolizes sales of Tesla automobiles. However, sales of vehicles in general are highly competitive and substitutable. Claiming that Tesla's abandonment of the franchise model is uncompetitive is similar to claiming that McDonald's refusal to allow Burger King to sell Big Macs is uncompetitive. You can't get a Big Mac at Burger King but you can get a near enough analogue that a claim of a monopoly is ludicrous.
Meanwhile Tesla claims that franchise laws are protectionist. Objectively this does seem to be the case, and with good historical reason. Auto franchise laws were established in order to prevent manufacturers from driving local dealers out of business with lower prices. The rationale behind these laws do not seem to currently apply in Tesla's case. After all, Tesla has no dealers to drive out of business. However, laws are not enforced based on whether a historical rationale applies or not. The real question is, do franchise laws provide greater societal utility than Tesla's direct to consumer model?
Realistically the role of the auto dealership is rapidly becoming a relic. There is no need to hold a large inventory of cars in what amounts to a parking lot. Consumers would like the ability to test an automobile. However, dealerships serve little useful purpose beyond these test drives. What benefit does a sprawling dealership with several hundred cars offer over a small show room with a couple dozen test vehicles? Why force consumers to choose between a subset of pre-made cars when one can be built to order nearly as easily? It's hard to find any way in which Tesla's model isn't better and cheaper for buyers. Note that this rationale applies only to new car sales (unless someone discovers a way to make used cars made to order).
Unfortunately simply obliterating all new car dealerships would have vast economic ramifications. While consumers would benefit overall from cheaper, more customized automobiles, thousands of jobs would be lost. Despite what seem to be clear efficiency gains the dealership industry needs to be allowed to unravel slowly and naturally to prevent an economic shock. However, this won't happen as long as protectionist laws prevent manufacturers from selling directly to buyers.
Inevitably the direct to consumer model will triumph as long as more protectionist laws are not enacted. Consumers in states with protectionist legislation will simply purchase in other states or online. However, until then the slow decline of the auto dealer is probably the best way to unwind the industry.
That's all for this week. Until next time stay safe and rationale.
Tesla is reimagining nearly every aspect of the automotive. Everything from manufacturing to sales has been reconsidered from the ground up. Up until now Tesla's innovations have been primarily restricted to the reasonably wealthy. However, it's not difficult to imagine many of their innovations propagating to the majority of automotive sales.
To most people Tesla's largest innovation is likely the manufacture of completely electric, long range, luxury cars. However, the less obvious disruption of their sales philosophy is equally game changing. Tesla sells their cars direct to consumers, rather than through a franchise like most auto makers.
Critics claim that the direct to consumer model stifles competition as Tesla can charge whatever prices they wish. It's difficult to view this idea as anything more than ridiculous. Granted, Tesla monopolizes sales of Tesla automobiles. However, sales of vehicles in general are highly competitive and substitutable. Claiming that Tesla's abandonment of the franchise model is uncompetitive is similar to claiming that McDonald's refusal to allow Burger King to sell Big Macs is uncompetitive. You can't get a Big Mac at Burger King but you can get a near enough analogue that a claim of a monopoly is ludicrous.
Meanwhile Tesla claims that franchise laws are protectionist. Objectively this does seem to be the case, and with good historical reason. Auto franchise laws were established in order to prevent manufacturers from driving local dealers out of business with lower prices. The rationale behind these laws do not seem to currently apply in Tesla's case. After all, Tesla has no dealers to drive out of business. However, laws are not enforced based on whether a historical rationale applies or not. The real question is, do franchise laws provide greater societal utility than Tesla's direct to consumer model?
Realistically the role of the auto dealership is rapidly becoming a relic. There is no need to hold a large inventory of cars in what amounts to a parking lot. Consumers would like the ability to test an automobile. However, dealerships serve little useful purpose beyond these test drives. What benefit does a sprawling dealership with several hundred cars offer over a small show room with a couple dozen test vehicles? Why force consumers to choose between a subset of pre-made cars when one can be built to order nearly as easily? It's hard to find any way in which Tesla's model isn't better and cheaper for buyers. Note that this rationale applies only to new car sales (unless someone discovers a way to make used cars made to order).
Unfortunately simply obliterating all new car dealerships would have vast economic ramifications. While consumers would benefit overall from cheaper, more customized automobiles, thousands of jobs would be lost. Despite what seem to be clear efficiency gains the dealership industry needs to be allowed to unravel slowly and naturally to prevent an economic shock. However, this won't happen as long as protectionist laws prevent manufacturers from selling directly to buyers.
Inevitably the direct to consumer model will triumph as long as more protectionist laws are not enacted. Consumers in states with protectionist legislation will simply purchase in other states or online. However, until then the slow decline of the auto dealer is probably the best way to unwind the industry.
That's all for this week. Until next time stay safe and rationale.
Monday, March 3, 2014
Publishing PLoS
One of my favorite journals is the Public Library of Science. PLoS is an open access, online journal which has consistently gone above and beyond in an effort to bring science to the general public. Today PLoS took that effort one step further by requiring authors make data sets available on request after publication. Essentially authors must provide access to a minimal data set (the data utilized to substantiate the studies results) or a valid exclusionary excuse in order to be published in PLoS.
As an advocate of open access in scientific research I was thrilled to hear of this policy change. However, even I was aware that some data sets should remain private either due to moral concerns or difficulty in delivery. Luckily PLoS has already addressed the concerns that I and many others have.
Some research generates such vast amounts of data that retaining it (much less supplying it) is simply financially and logistically burdensome. The Large Hadron Collider for example generates upwards of three hundred and fifty gigabytes of data every second. Over ninety percent of this data is immediately discarded before significant analysis is performed. After further filtering over ninety-nine point nine nine percent of all data is lost forever. Only a minuscule portion of data is ever seen by researchers, much less the general public. Obviously it is ridiculous for the LHC to provide full data sets for every result they publish.
PLoS of course agrees. It's important to note that researchers must only publish minimal data sets, or the data that is required to derive their presented results. For example, the LHC would only need to provide access to the data for collisions directly relevant to a particular conclusion. Yet, even then the amount of data required is massive. Luckily PLoS is willing to work with authors when such difficulties arise. They point to a variety of services such as Dryad, Genbank, and clinicaltrials.gov as possible solutions for large data sets. If none of those are adequate to an author's needs PLoS will work with them on a one on one basis to make sure everyone's interests are served.
At times the ability to supply data sets isn't a concern but the moral repercussions are questionable. Should ecologists studying valuable endangered species be required to reveal their habitats locations? Should patients in clinical trials have their personal details revealed? Should the genetic sequences for dangerous pathogens be freely available? It's difficult to make an argument that any of these questions should be unequivocally answered "yes". PLoS acknowledges that at times ethical and legal concerns make data publication untenable and will work with authors when this is the case.
Despite these reasonable exceptions PLoS's policy changes are a monumental step in the right direction for open access. I can only hope that they continue to push the frontier of scientific publication forward in the future.
That's all for this week. Until next time stay safe and rationale.
As an advocate of open access in scientific research I was thrilled to hear of this policy change. However, even I was aware that some data sets should remain private either due to moral concerns or difficulty in delivery. Luckily PLoS has already addressed the concerns that I and many others have.
Some research generates such vast amounts of data that retaining it (much less supplying it) is simply financially and logistically burdensome. The Large Hadron Collider for example generates upwards of three hundred and fifty gigabytes of data every second. Over ninety percent of this data is immediately discarded before significant analysis is performed. After further filtering over ninety-nine point nine nine percent of all data is lost forever. Only a minuscule portion of data is ever seen by researchers, much less the general public. Obviously it is ridiculous for the LHC to provide full data sets for every result they publish.
PLoS of course agrees. It's important to note that researchers must only publish minimal data sets, or the data that is required to derive their presented results. For example, the LHC would only need to provide access to the data for collisions directly relevant to a particular conclusion. Yet, even then the amount of data required is massive. Luckily PLoS is willing to work with authors when such difficulties arise. They point to a variety of services such as Dryad, Genbank, and clinicaltrials.gov as possible solutions for large data sets. If none of those are adequate to an author's needs PLoS will work with them on a one on one basis to make sure everyone's interests are served.
At times the ability to supply data sets isn't a concern but the moral repercussions are questionable. Should ecologists studying valuable endangered species be required to reveal their habitats locations? Should patients in clinical trials have their personal details revealed? Should the genetic sequences for dangerous pathogens be freely available? It's difficult to make an argument that any of these questions should be unequivocally answered "yes". PLoS acknowledges that at times ethical and legal concerns make data publication untenable and will work with authors when this is the case.
Despite these reasonable exceptions PLoS's policy changes are a monumental step in the right direction for open access. I can only hope that they continue to push the frontier of scientific publication forward in the future.
That's all for this week. Until next time stay safe and rationale.
Friday, February 21, 2014
Eye of the Tiger
A bit of a disclaimer about this week's posting. I'm in no way endorsing or promoting the product I'm about to discuss. I have not personally used or purchased it. In fact I encourage anyone to approach a product that uses the phrase "brain training" with a healthy dose of skepticism. All that said, if results hold up it's interesting and thus worth discussion.
Recently a new app has been generating some buzz in neuroscience circles. Dr. Aaron Seitz has developed UltimEyes, a program which claims to improve visual acuity by training the visual cortex. Seitz recently published a study wherein baseball players seemingly exhibited significant improvements in vision (and subsequently hitting).
First the bad news. Although Seitz is a serious researcher with a prominent background in the discipline his UltimEyes study was preliminary at best. Due to a variety of factors (which reportedly center around requests originating from the player's coach) the study was non-blinded, the treatment group was non-random and the control group received no placebo. Further, while vision improvements (as demonstrated by a standard eye chart) were clear and attributable to the study's treatment; hitting improvements (attributed largely by Seitz to improved vision) are easily explainable by other mechanisms. Essentially the study was conducted during the portion of the season that hitters (the treatment group) improve the most anyway whereas the control group (pitchers) rarely improve much at all.
The good news is that by all appearances the app does improve vision in otherwise healthy adults. It's important to note that the program does nothing to repair or improve the eye. If there is some physical cause of poor sight the program will not change that. However, reportedly by training the brain to process Gabor Stimuli more efficiently individuals can see (pun intended) significant improvements in eyesight.
There has been somewhat of a fad recently in so called "brain training" workouts. The goal is to keep the mind agile and stave off the eventual cognitive decline caused by aging. Most of these workouts are essentially scams. Keeping your mind active is an important part of overall physical health. However, these marketed programs generally have no remarkable science behind them. Any puzzle, discussion, or even meditation which sufficiently stimulates the mind would have the same effect as an expensive brain training program. Furthermore, all o the above can be found far cheaper than a brain trainer.
Seitz's work on the other hand is rather novel. The app presents the visual cortex with stimuli which seem to improve it's efficiency. Stimuli which you are unlikely to encounter on your own. Unlike brain trainers which simply sell simple puzzles at huge profits, Seitz seems to be selling something you can't easily get for free in your day to day life. Undoubtedly free copycats will arise which are similarly effective (the app is not complex) but the point is Seitz seems to be peddling more than just snake oil.
Undoubtedly the research needs to be taken further before anyone can claim Seitz's work is worth your time (much less your money) but it's a promising and interesting exploration into how neuroscience can improve individual's day to day lives. Next up could be improving individuals ability to hear a certain voice in a crowded environment or curing tone deafness.
A final note. By all reports the app is not particularly well made or functional. Many users report the app crashes often or doesn't properly allow logins. Again, I do not encourage anyone to purchase the app, only to explore the science behind it.
That's all for this week. Until next time stay safe and rationale.
Recently a new app has been generating some buzz in neuroscience circles. Dr. Aaron Seitz has developed UltimEyes, a program which claims to improve visual acuity by training the visual cortex. Seitz recently published a study wherein baseball players seemingly exhibited significant improvements in vision (and subsequently hitting).
First the bad news. Although Seitz is a serious researcher with a prominent background in the discipline his UltimEyes study was preliminary at best. Due to a variety of factors (which reportedly center around requests originating from the player's coach) the study was non-blinded, the treatment group was non-random and the control group received no placebo. Further, while vision improvements (as demonstrated by a standard eye chart) were clear and attributable to the study's treatment; hitting improvements (attributed largely by Seitz to improved vision) are easily explainable by other mechanisms. Essentially the study was conducted during the portion of the season that hitters (the treatment group) improve the most anyway whereas the control group (pitchers) rarely improve much at all.
The good news is that by all appearances the app does improve vision in otherwise healthy adults. It's important to note that the program does nothing to repair or improve the eye. If there is some physical cause of poor sight the program will not change that. However, reportedly by training the brain to process Gabor Stimuli more efficiently individuals can see (pun intended) significant improvements in eyesight.
There has been somewhat of a fad recently in so called "brain training" workouts. The goal is to keep the mind agile and stave off the eventual cognitive decline caused by aging. Most of these workouts are essentially scams. Keeping your mind active is an important part of overall physical health. However, these marketed programs generally have no remarkable science behind them. Any puzzle, discussion, or even meditation which sufficiently stimulates the mind would have the same effect as an expensive brain training program. Furthermore, all o the above can be found far cheaper than a brain trainer.
Seitz's work on the other hand is rather novel. The app presents the visual cortex with stimuli which seem to improve it's efficiency. Stimuli which you are unlikely to encounter on your own. Unlike brain trainers which simply sell simple puzzles at huge profits, Seitz seems to be selling something you can't easily get for free in your day to day life. Undoubtedly free copycats will arise which are similarly effective (the app is not complex) but the point is Seitz seems to be peddling more than just snake oil.
Undoubtedly the research needs to be taken further before anyone can claim Seitz's work is worth your time (much less your money) but it's a promising and interesting exploration into how neuroscience can improve individual's day to day lives. Next up could be improving individuals ability to hear a certain voice in a crowded environment or curing tone deafness.
A final note. By all reports the app is not particularly well made or functional. Many users report the app crashes often or doesn't properly allow logins. Again, I do not encourage anyone to purchase the app, only to explore the science behind it.
That's all for this week. Until next time stay safe and rationale.
Friday, February 14, 2014
Socially Responsible Investing
Before reading further take a moment to consider, what is the purpose of investment?
Most people probably would reply that investment is a means to acquire wealth and income. On the individual level they're most likely correct. Few people invest with hopes of losing their nest egg. But on a social level what purpose does investment serve?
The answer likely isn't terribly intuitive for non-economists. Investors, through their self interested wealth creation should be allocating capital to firms that will use it most efficiently. For example, if Bill's company has a fantastic new software product that people will find useful he should be able to sell it for a decent profit. This prospective profit should allow Bill to attract investors who will provide the funds for Bill to fully develop and distribute his product. The end result is that Bill and his investors become wealthier and society benefits from a product that improves everyone's lives. Everybody wins and everyone is happy.
This pattern has been followed in various forms for centuries. Spice traders would provide funds for caravans to travel eastward, buy spices and return them to their homeland. Again, society benefits from the spices they desire and the investors and firms profit. Kings and Queens would provide ships and crew for explorers mapping far off continents. The explorers opened up new lands for society to develop and returned with new riches to fill the coffers of their investors. To generalize the pattern, investors provide capital to firms or individuals who then transform the capital into a good which society values more than the capital. The firm collects the excess valuation (profits) and passes it on in part or whole to the investors.
Contrast this pattern with gambling. Gambling, much like investing, has the same goal to the individual, wealth accumulation (although hopefully the ultimate goal is entertainment rather than income). However, gambling provides no benefit to society. It is simply the semi-random exchange of funds between two parties, the gambler and the house. Although a gambler and investor both give funds to another party in the hope of great return, the gambler has no thought to a casino using funds productively or efficiently. An investor meanwhile will only provide funds to a firm which he believes has a profitable purpose and effective means to accomplish it's purpose.
This brings us then to the concept of speculation, the investment in a firm or commodity simply due to the belief that the equity will appreciate in value. An investor who buys iron with the belief that iron will become more valuable due to possible shortages performs no service to society. Ultimately iron mining firms will seek investment to expand operations and thus fulfill societies iron needs. But the investors that buy shares in the firms will be allocating capital optimally (from a societal perspective) whereas those that buy iron accomplish nothing but driving prices higher. Both (assuming an increase in iron prices) will profit, but those that buy iron will only benefit themselves.
Speculation is at best attempting to outguess the market. Whereas investment is the efficient allocation of funds in a free market. It's perfectly reasonable to invest in facebook because you believe the management will use the funds productively. However, investing in facebook simply because you believe others will drive up the price accomplishes nothing but the gain or loss of personal wealth. Of course, on the individual level most people are more than satisfied with "just" the gain of personal wealth. Still, in an ideal world investors would always gain profit at the same time they're helping keep the economy moving forward.
That's all for this week. Until next time stay safe and rationale.
Friday, February 7, 2014
Man vs Mechanization
Predictions that automation will supplant human workers are nothing new. Indeed, in many industries machines have already taken over jobs previously performed by low skilled workers. Yet, overall the inevitable rise of the machines has fallen rather short of most futurists predictions. Still, over a long enough time line it's difficult to imagine how automation doesn't supplant most low skill jobs. After all, machines in general only grow cheaper while human labor often rises in cost.
Given the machines eventual workforce dominance it's an interesting exercise to determine which jobs are reasonably safe from robotic theft. To this end let's define what machines do well and where they fall short.
Given the machines eventual workforce dominance it's an interesting exercise to determine which jobs are reasonably safe from robotic theft. To this end let's define what machines do well and where they fall short.
- Machines perform routine tasks quickly and cheaply.
- Machines work without fatigue or distraction.
- The cost of a machine is heavily front loaded and ends when a machine is decommissioned.
- Machines do not react to novelty with adaptive behavior.
- Machines are generally either fully operational or non productive.
- Machines do not exceed their designed purpose.
Points one through three are for the most part advantages machines have over people while points four to six are deficits. Machines are generally more productive, at less cost and require no payment when they're not working (such as retirement benefits or vacation time). These characteristics make them generally preferable workers when suited for the task. Points four through six roughly define the tasks ill suited to machine labor.
Machines do not react to novelty with adaptive behavior. Roughly speaking this means tasks which deal with highly variable inputs are often ill suited to automation. In customer service for example workers must engage with a customer who may have any number of complaints (both reasonable and unreasonable) and attempt to satisfy the individual. Contrarily, the only input of importance for a cashier is barcodes and currency, two largely uniform inputs a machine can easily work with. Thus self check out lanes in stores make a great deal of sense while self customer service counters do not. Interestingly self return counters in stores would make sense (after all it's just a reverse checkout) if not for the likelihood of deception on the part of customers. The essence of this point is that we have yet to create machines which make decisions wisely. A task must be routine and approachable algorithmically for machine labor to be efficient. Any job which consists primarily of judgement and decisions is likely safe from a machine takeover for the foreseeable future. This includes governance, law enforcement, any profession requiring extensive human interaction (note that just because a profession currently involves human interaction does not mean it requires it), law, insurance, any profession involving diagnosis (both health and maintenance), inspection personnel, and management.
Machines are generally either fully operational or non productive. There is the possibility of performance degradation over time, but machines are a far more binary system than people. When a part of a machine fails it's very likely to stop completely. People are far more resilient and predictable when it comes to component failures. If a robot doctor (an independent robot surgeon, not a tool used by a human surgeon) had a component seize during a surgery the results would likely be life ending. However, a human doctor with a hand cramp simply pauses, stretches out his hand for a moment and continues with the procedure. Situations like these make machine labor unsuitable in scenarios where failure is unacceptable. If there were a profession who's duties included "push this button once an hour or everyone on earth dies" we'd likely build a machine to push the button, but you can also be sure at least one human would be there as well to make sure it was done. Professions made safe by this point are: many healthcare professionals, large construction project jobs, pilots, and any profession which puts lives or large amounts of money at risk if done incorrectly.
Machines do not exceed their designed purpose. Essentially this amounts to machines are not very creative. They do not invent, innovate, create art or entertainment. Any profession which relies primarily on the imagination of the human mind is for the moment unlikely to be taken by a robot. This includes artists, actors advertisers, academics, inventors, engineers, programmers, design, everything hollywood, and essentially any scientific pursuit.
It's important to note a key difference between machine automation that replaces a job and machine automation that assists a job. Arguably on a plane the autopilot is a machine that does most of the work. However, it is unlikely that we'll eliminate the profession of pilot any time soon. Similarly, many surgeons are using machines which reduce the invasiveness of surgeries and speed recovery times. But machines which conduct major surgeries without human input are probably not just around the corner.
Undoubtedly there are a number of professions not explicitly mentioned which are equally safe from the automation revolution. However, much of manufacturing has already been supplanted by robotic workers and the trend continues. The next wave is likely to be the elimination of many service industry jobs as cheap and easy to use interfaces replace cashiers and restaurant workers. If half your job involves handing someone a receipt be on the look out for R2-D2 coming to take your job. Though as dire as that sounds, keep in mind even the self check out lanes at grocery stores still have a real person there making sure things go smoothly.
That's it for this week. Until next time stay safe and rationale.
Friday, January 31, 2014
Bounded Rationality
It has occurred to me that it would be irresponsible to not address the namesake of this blog at some point. Therefore, in brief here is an explanation of bounded rationality.
Economists generally prefer to view people as perfectly rational individuals. A perfectly rational being is predictable and easily modeled. However, at some point even the most die hard academic had to admit that people make suboptimal, or even downright poor decisions. The acknowledgement of this fact led to a number of changes in economics, the most influential being the rise of Homo Economicus and bounded rationality.
Homo Economicus is a theoretical tool economists use in creating models. He is a representation of humanity that is perfectly rational, narrowly focused and purely self interested. He maximizes personal utility at every opportunity and never makes foreseeable mistakes. Essentially Homo Economicus is a theoretical human who's behavior is easily defined in nearly any situation. Economists all acknowledge that Homo Economicus is a fiction. However, in many cases his behavior doesn't diverge all that much from what occurs in reality. Thus we keep him around in a back closet and shamefully drag him out whenever we need to create a new model of auction pricing, consumer behavior or whatever we're working on that week.
The other concept, and namesake of this blog, is bounded rationality. Bounded rationality is simply the idea that although humans are basically rational, a great deal of obstacles get in the way of that rationality. Emotion, time constraint, lack of information, and inability to process sufficient information all are major obstacles to making optimal decisions.
For example, suppose Alice was shopping for health insurance. She has four choices with the following information: Plan A has a lifetime cost of $6000 and a lifetime benefit of $5000, Plan B has a lifetime cost of $8000 and a lifetime benefit of $10,000, Plan C has a lifetime cost of $3000 and a lifetime benefit of $1000, and Plan D has a lifetime cost of $2000 and no lifetime benefit.
Given only this information it's easy to conclude that Plan B is the best. It's the only plan which provides a benefit that exceeds costs. However, Alice may not choose Plan B for a variety of reasons. The first and largest reason she may not choose Plan B is although we have defined future benefits Alice has no way to anticipate them so easily. She can reasonably approximate future costs of the plans but healthcare utilization is an unpredictable factor. Thus she can't know for sure what level of benefit each plan will provide. Here her lack of knowledge has limited her ability to make an optimal decision.
Imagine Alice was only given a short time to choose her health insurance. This time constraint would very likely further impede her ability to choose wisely. Rather than conducting comprehensive research on costs and benefits Alice would probably fall back on a heuristic shortcut such as, "choose the cheapest plan that meets my anticipated minimum needs." While this will often times lead to a selection which is "good enough" it's a decision method which has discarded the goal of optimal selection. Thus time constraint has led to a non optimal selection method.
Alice may have had a difficult week at work and is too stressed to make a careful selection. She simply wants the decision made so she can put it behind her so she selects mostly at random. Here emotion has greatly impaired Alice's rationality.
Finally, suppose each plan had innumerable payment schedules, reimbursement plans, copays, deductibles, and similar minutiae which populate health insurance plans. Alice may very well not be capable of processing all of that information and making intelligent comparisons between plans. In this case Alice's inability to process information adequately puts her in a situation where her decision is more an informed guess than an optimal selection.
Clearly there are many factors which may impair a person's rational thinking. By saying individuals exhibit bounded rationality we are simply stating that a person is generally rational within the limitations of their knowledge and abilities. When that knowledge is incomplete or those abilities insufficient to the task they may exhibit behavior which is clearly suboptimal from the standpoint of Homo Economicus.
That's all for this week. Until next time stay safe and rational.
Economists generally prefer to view people as perfectly rational individuals. A perfectly rational being is predictable and easily modeled. However, at some point even the most die hard academic had to admit that people make suboptimal, or even downright poor decisions. The acknowledgement of this fact led to a number of changes in economics, the most influential being the rise of Homo Economicus and bounded rationality.
Homo Economicus is a theoretical tool economists use in creating models. He is a representation of humanity that is perfectly rational, narrowly focused and purely self interested. He maximizes personal utility at every opportunity and never makes foreseeable mistakes. Essentially Homo Economicus is a theoretical human who's behavior is easily defined in nearly any situation. Economists all acknowledge that Homo Economicus is a fiction. However, in many cases his behavior doesn't diverge all that much from what occurs in reality. Thus we keep him around in a back closet and shamefully drag him out whenever we need to create a new model of auction pricing, consumer behavior or whatever we're working on that week.
The other concept, and namesake of this blog, is bounded rationality. Bounded rationality is simply the idea that although humans are basically rational, a great deal of obstacles get in the way of that rationality. Emotion, time constraint, lack of information, and inability to process sufficient information all are major obstacles to making optimal decisions.
For example, suppose Alice was shopping for health insurance. She has four choices with the following information: Plan A has a lifetime cost of $6000 and a lifetime benefit of $5000, Plan B has a lifetime cost of $8000 and a lifetime benefit of $10,000, Plan C has a lifetime cost of $3000 and a lifetime benefit of $1000, and Plan D has a lifetime cost of $2000 and no lifetime benefit.
Given only this information it's easy to conclude that Plan B is the best. It's the only plan which provides a benefit that exceeds costs. However, Alice may not choose Plan B for a variety of reasons. The first and largest reason she may not choose Plan B is although we have defined future benefits Alice has no way to anticipate them so easily. She can reasonably approximate future costs of the plans but healthcare utilization is an unpredictable factor. Thus she can't know for sure what level of benefit each plan will provide. Here her lack of knowledge has limited her ability to make an optimal decision.
Imagine Alice was only given a short time to choose her health insurance. This time constraint would very likely further impede her ability to choose wisely. Rather than conducting comprehensive research on costs and benefits Alice would probably fall back on a heuristic shortcut such as, "choose the cheapest plan that meets my anticipated minimum needs." While this will often times lead to a selection which is "good enough" it's a decision method which has discarded the goal of optimal selection. Thus time constraint has led to a non optimal selection method.
Alice may have had a difficult week at work and is too stressed to make a careful selection. She simply wants the decision made so she can put it behind her so she selects mostly at random. Here emotion has greatly impaired Alice's rationality.
Finally, suppose each plan had innumerable payment schedules, reimbursement plans, copays, deductibles, and similar minutiae which populate health insurance plans. Alice may very well not be capable of processing all of that information and making intelligent comparisons between plans. In this case Alice's inability to process information adequately puts her in a situation where her decision is more an informed guess than an optimal selection.
Clearly there are many factors which may impair a person's rational thinking. By saying individuals exhibit bounded rationality we are simply stating that a person is generally rational within the limitations of their knowledge and abilities. When that knowledge is incomplete or those abilities insufficient to the task they may exhibit behavior which is clearly suboptimal from the standpoint of Homo Economicus.
That's all for this week. Until next time stay safe and rational.
Thursday, January 23, 2014
Researcher Integrity
Two years ago Adam Wilkinson was forced to retract a prominent fMRI study regarding language processing due to suspected fraud. What follows is an interview conducted in November of last year in which he discusses that study.
Interviewer: Thank you for speaking with me Adam. Obviously the retraction of your language processing paper in 2012 was a significant career setback. Some people allege that you knowingly manipulated data in order to get the results that would merit publication. How do you respond to those allegations?
Adam: At no time did I modify data in any way in order to manipulate results.
Interviewer: So no data was changed at any point?
Adam: That is correct.
Interviewer: Was any data discarded for any reason?
Adam: Some data sets were discarded as is routine during a study of this nature.
Interviewer: And what were the reasons for ignoring those data sets?
Adam: In some cases post scan surveys revealed characteristics of subjects that made them unsuitable for the experiment. Other data sets were removed due to excessive head motion or an inability to normalize their data with our chosen methods.
Interviewer: So in other words some people moved around too much or had previously unknown neuorological conditions?
Adam: Yes. Or their head orientation or brain shape did not easily conform with a standard fitting model.
Interviewer: If you had included those data sets would your results have remained the same?
Adam: To include those data sets we would have had to adjust our analytic framework. It's possible that doing so would have altered the end results.
Interviewer: Several labs have attempted to replicate your findings but have as yet been unable. Do you believe your initial study to be in error?
Adam: In the time since publication it has become evident that our initial publication was likely incorrect. Unfortunately there is always a small chance of a false positive in research and this time we got unlucky.
Interviewer: Your critics say that in this case you made your own luck. They claim you selectively eliminated data and chose analytical methods which would lead to a positive result. In particular Nick Harrison has been quoted as saying, "Nearly any data set and analytic method except those used would have lead to a negative result." What's your response to that?
Adam: No doubt there are statistical techniques which would change the results of the study. I reject the idea that I chose my method in order to manipulate the result.
Interviewer: Was your method of analysis determined before data was collected?
Adam: It was not.
Interviewer: Was your hypothesis preregistered?
Adam: It was not.
Interviewer: So it would have been possible for you to choose a protocol and data set combination that would lead to a positive result.
Adam: Possible, but it did not occur.
Interviewer: How many subjects were included in this study?
Adam: Due to budget limitations we were only able to include fourteen participants.
Interviewer: How many participants had their data discarded for any reason?
Adam: Two.
Interviewer: Is twelve subjects enough to reach significant conclusions?
Adam: Due to the high cost of fMRI studies twelve subjects is not unusual. It's possible that it led to an under-powered study but it is not unusual.
Interviewer: Does a small group of participants make it easier or more difficult to change results by altering analytic protocols and discarding data sets.
Adam: Experiments with few participants are altered by a greater degree when a small number of subject's data is discarded. In that way it would make altering the results easier.
Interviewer: Were you under pressure to publish something significant at the time of publication?
Adam: In academic research there is always a pressure to publish.
Interviewer: So there was no unusual pressure at that time?
Adam: The high cost of fMRI studies makes all such studies a significant investment. It would have been disappointing to have invested so much and not discovered anything of consequence. Beyond that, no.
Interviewer: Some have claimed that you didn't initially share your raw data in an effort to cover up fraud. Was that the case?
Adam: Data sharing in the field is a rarity more than the norm. As soon as the publication was called into question I made the raw data freely available to anyone who requested it.
Interview
Interviewer: Thank you for speaking with me Adam. Obviously the retraction of your language processing paper in 2012 was a significant career setback. Some people allege that you knowingly manipulated data in order to get the results that would merit publication. How do you respond to those allegations?
Adam: At no time did I modify data in any way in order to manipulate results.
Interviewer: So no data was changed at any point?
Adam: That is correct.
Interviewer: Was any data discarded for any reason?
Adam: Some data sets were discarded as is routine during a study of this nature.
Interviewer: And what were the reasons for ignoring those data sets?
Adam: In some cases post scan surveys revealed characteristics of subjects that made them unsuitable for the experiment. Other data sets were removed due to excessive head motion or an inability to normalize their data with our chosen methods.
Interviewer: So in other words some people moved around too much or had previously unknown neuorological conditions?
Adam: Yes. Or their head orientation or brain shape did not easily conform with a standard fitting model.
Interviewer: If you had included those data sets would your results have remained the same?
Adam: To include those data sets we would have had to adjust our analytic framework. It's possible that doing so would have altered the end results.
Interviewer: Several labs have attempted to replicate your findings but have as yet been unable. Do you believe your initial study to be in error?
Adam: In the time since publication it has become evident that our initial publication was likely incorrect. Unfortunately there is always a small chance of a false positive in research and this time we got unlucky.
Interviewer: Your critics say that in this case you made your own luck. They claim you selectively eliminated data and chose analytical methods which would lead to a positive result. In particular Nick Harrison has been quoted as saying, "Nearly any data set and analytic method except those used would have lead to a negative result." What's your response to that?
Adam: No doubt there are statistical techniques which would change the results of the study. I reject the idea that I chose my method in order to manipulate the result.
Interviewer: Was your method of analysis determined before data was collected?
Adam: It was not.
Interviewer: Was your hypothesis preregistered?
Adam: It was not.
Interviewer: So it would have been possible for you to choose a protocol and data set combination that would lead to a positive result.
Adam: Possible, but it did not occur.
Interviewer: How many subjects were included in this study?
Adam: Due to budget limitations we were only able to include fourteen participants.
Interviewer: How many participants had their data discarded for any reason?
Adam: Two.
Interviewer: Is twelve subjects enough to reach significant conclusions?
Adam: Due to the high cost of fMRI studies twelve subjects is not unusual. It's possible that it led to an under-powered study but it is not unusual.
Interviewer: Does a small group of participants make it easier or more difficult to change results by altering analytic protocols and discarding data sets.
Adam: Experiments with few participants are altered by a greater degree when a small number of subject's data is discarded. In that way it would make altering the results easier.
Interviewer: Were you under pressure to publish something significant at the time of publication?
Adam: In academic research there is always a pressure to publish.
Interviewer: So there was no unusual pressure at that time?
Adam: The high cost of fMRI studies makes all such studies a significant investment. It would have been disappointing to have invested so much and not discovered anything of consequence. Beyond that, no.
Interviewer: Some have claimed that you didn't initially share your raw data in an effort to cover up fraud. Was that the case?
Adam: Data sharing in the field is a rarity more than the norm. As soon as the publication was called into question I made the raw data freely available to anyone who requested it.
Discussion
Luckily Adam Wilkinson and his research are fictional. He was made up in order to demonstrate the ease by which a researcher can guide their studies results.
In most fMRI studies some data is discarded. It's an unfortunate side effect of using such a precise instrument on subjects who have difficulty remaining still. An unscrupulous researcher, such as Adam, can determine which data sets will move results towards a predetermined end when eliminated.
Similarly there are dozens of analytical methods which a researcher can use to perform statistical comparisons. Each of these methods will give slightly different results and some will almost certainly give a positive result. Here's an example.
Let's say Adam had thirty people scheduled to receive scans. After five people have been scanned he begins to check his data between scans to see if he has a statistically significant result. On the twelfth scan the data barely tips into the statistically significant range and Adam cancels the remaining scans. The data for scans 1-11 didn't show what Adam was looking for but due to an anomaly on the 12th scan Adam got lucky and the data worked out for him. Had scans 13-30 been conducted the data would have returned to the mean and Adam wouldn't have the result he sought. Since Adam didn't preregister his hypothesis or protocol he can peek at the data for any number of statistical tests and stop whenever any of them dip into the range of significance. Unfortunately after the fact there's really no way to tell if Adam did this or not.
FMRI data is by it's nature very "noisy". It requires quite a bit of pre-processing before it can be analyzed in a meaningful fashion. How this pre-processing is done can wildly change results. Adam can adjust his pre-processing parameters in a large variety of ways in order to manipulate his results. As long as Adam keeps his parameters reasonable no one will question this aspect of his research.
FMRI studies look at a particular region of interest in the brain. If Adam is conducting research on language processing than there's a general region that should be his focus. But the exact defined area of that region is a matter of Adam's discretion. Does he outline a region that's just a little more anterior than usual thus including a bit of a the visual cortex? If so he can show activation in his region of interest that really is attributable to the visual cortex, not any sort of language processing. This sort of deception is detectable but only with close inspection by a proficient expert. Even then Adam could present it as a simple error that wouldn't change results.
What comparison does Adam make? Does he compare individuals to their own baseline scan or to a group normalized scan? Does he group subjects into male and female and conduct comparisons within those groups only? Perhaps divisions are created based on gender and age or on age alone. Each comparison made allows Adam another chance at a statistically significant result. Since he didn't state his hypothesis before starting the experiment he can test as many things as he can imagine until one by chance comes up as significant.
It's easy to see that given all these factors a researcher has thousands of possible end results. The sheer volume of possibilities nearly guarantees that some null hypothesis can be rejected and the experimenter can find a publishable result. Most importantly all of this can be done without "changing" any of the data.
Adam's answers in the interview would in most cases be typical of a researcher in his field. Small studies, without preregistered protocols, and with hidden data sets are routine in fMRI research. Unfortunately this presents a ripe opportunity for researchers to manipulate their protocols in order to meet publishing pressures.
Many researchers would argue that fMRI research is by it's nature exploratory and observational. However, it is dishonest to make such a claim while at the same time hiding behind the all mighty p-value. Either you are hypothesis testing, in which case a p-value is valid and you know your hypothesis before hand or you're conducting an observational study and a p-value isn't relevant. In this case you can't have your cake and eat it too.
That's all for this week. Until next time stay safe and rationale.
Saturday, January 11, 2014
The Economics of Cutthroat Kitchen
During the holiday season a coworker insisted I watch a new Food Network show called Cutthroat Kitchen. The show follows a standard chef game show format in which four chefs must cook a given menu item. A guest judge then determines which chef's dish was the worst and that chef is eliminated. The show proceeds through three rounds, eliminating one chef a round until the final chef is crowned the victor.
Here is the interesting twist meant to keep audiences coming back for more. The chefs are given their prize ($25,000) at the start of the show and may spend that money in auctions during each round. Auctions are generally for sabotages which can be given to other contestants to make their life more difficult. For example, if the item to be cooked was meatloaf and mashed potatoes a sabotage might be to force another contestant to make mashed potatoes with potato peels or to make meatloaf with unground meat. When losers are eliminated they return their prize money to the host whereas the winner keeps whatever they didn't spend over the course of the show.
As someone who cooks frequently the show has no educational value. The techniques or tricks the chefs use aren't discussed. Instead the focus is put on how a chef deals with the obstacles heaped upon him. While amusing, it's unlikely your average viewer will ever need to cook a pancake on top of a tin can. Thus, unless your cookware is stolen by the international culinary mafia you probably won't learn much.
As an economist however, the auctions are fascinating. It's rare to see so blatantly how bad we are at making financial decisions under pressure. Here is a fictional, but representative, example of how cutthroat kitchen often goes. Remember each contestant has $25,000 to spend in auctions. Any amount they don't spend will be their prize if they win.
Allen, Beth, Candice and Dave arrive at the set. They're told they'll be cooking a pizza. As usual they have thirty minutes to complete the task after they grab their ingredients from the shows pantry. Everyone scrambles to grab the items they'll need to make a crust and sauce as well as some cheese and toppings to cover the pizza with.
Shortly after cooking begins the host starts to auction off sabotages. The first auction (of an expected four) is for the ability to force another contestant to make their sauce out of unripe tomatoes. Beth and Dave are both terrified of having to use unripe tomatoes and so bid against each other until Beth wins the auction for $5500. She awards the tomatoes to Candice.
Auction two is won by Dave who forces Allen to have to grate his own cheese from a block rather than using preshredded. Auction three is won again by Dave who forces Beth to choose from some common ice cream toppings instead of pizza toppings for his pizza. Finally auction four is won by Candice who gets revenge on Beth by making her cook her pizza on a griddle instead of in an oven.
The end result is Allen has to grate his cheese from a block. Beth has to use ice cream toppings instead of pizza toppings and cook her pizza on a griddle. Candice has to use unripe tomatoes in her sauce and Dave is unsabotaged. Additionally, Allen has spent nothing, Beth is down $5500, Candice has spent $5000, and Dave has spent $10,000. Judging commences and Candice is eliminated as the judges felt her pizza was the least delicious.
Round two commences and Dave is eliminated. Allen and Beth remain with $22,000 and $18,000 respectively. After the final round Beth is crowned the victor with winnings of $10,500.
Using this fictitious series of events we can point to a number of poor decisions contestants made along the way. In the first round Beth and Dave were willing to spend a great deal in order to avoid getting the under ripe tomato sabotage. This is a very common behavior as expressed by contestants. They often are willing to bid on an auction just to guarantee they are not stuck with the sabotage. However, in the first round losing an auction generally only costs a contestant a one third chance of being hit with the penalty (as the winner could award it to any of the three other contestants). Given this probability the prices contestants pay is borderline absurd. Contestants routinely bid as if losing an auction is a guarantee of receiving a penalty, a belief which is clearly incorrect.
Another oddity is that contestants are often most willing to spend during the first round of the contest. Bids of $4000 to $6000 are not uncommon in the first round while in rounds two and three items will routinely sell for under $3000. This is partially attributable to the fact that contestants that are loose with their money quickly deplete it or are eliminated thus leaving only more reluctant bidders with the assets needed for such lofty bids. Still, it's difficult to understand why these big spenders expend so many resources so early, thus leaving themselves vulnerable in later rounds where they're more likely to be hit with sabotages. A possible, if unlikely, explanation is they're attempting to put themselves in a position of weakness so that other players will award sabotages to stronger opponents. However, putting oneself in a position of difficulty in order to avoid difficulty seems rather nonsensical.
The most bizarre behavior is when contestants spread penalties relatively evenly among the other contestants. Each round one person is eliminated. The logical behavior is as soon as a person receives a sabotage all other sabotages should be heaped upon them. This eliminates animosity in later rounds (as the contestant will have been eliminated) and is the best method to ensure continuation through the rounds.
It would seem there are two possibly strong methods of bidding intelligently on the show given logical behavior (which the contestants never seem to exhibit). Either win the first bid or do not bid at all. If you choose to win the first bid you then award it to another player who then is signaled as weak to the remaining players. If that player does not win the second auction it should logically be awarded to them as well and they are now in a position where they are at best as weak as the next weakest player if they win the next two auctions and award them both to the same person. Thus the sabotaged player is forced into winning the second auction or having a very high probability of elimination. This method is aggressive and potentially resource intensive if played against another player with the same strategy. However, when facing three passive players (as described below) the resource cost is low. The considerable downside is whoever you award the sabotage to essentially must bid against you in the second auction or risk the piled on sabotages. If they win the second auction they're most likely awarding their sabotage to the person who sabotaged them in the first auction (you). So while you may gain a cheap sabotage to use against someone else, you also likely gain an expensive sabotage used on you. Overall it is a gain against the player you sabotaged, but a loss against the two players who are uninvolved.
The second, and likely preferable strategy is far simpler and likely superior. Do not bid on the first auction. The winner only has a one third chance of awarding the sabotage to you. Two thirds of the time you will be able to keep all your money and receive no sabotage for the first round (as all sabotages should go to the first person sabotaged or very likely the person who sabotaged them). In the unfortunate event that you are chosen for the first sabotage you are put in a situation where you must make another player as weak as you, meaning you must start bidding and winning auctions to weaken them. However, with the exception of a likely higher cost of auctions two, three and possibly four in the first round as compared to auction one you are at no particular disadvantage against the player you choose to target. Overall your expected earnings increase a great deal as well as your position in later rounds of the contest.
It would be interesting to conduct interviews with contestants in order to establish their motivations for how they award sabotages. Certainly the tendency to spread the penalties around seems to indicate contestants think more of equity than how best to improve their odds of victory. Still, at times I wonder if at times their consideration amounts to no more than, "I don't want to be stuck with that. I'll buy it then give it away to someone more or less at random." Regardless of motivation, the show is an excellent demonstration of how not to think about strategic decisions.
That's it for this week. Until next time stay safe and rationale.
Here is the interesting twist meant to keep audiences coming back for more. The chefs are given their prize ($25,000) at the start of the show and may spend that money in auctions during each round. Auctions are generally for sabotages which can be given to other contestants to make their life more difficult. For example, if the item to be cooked was meatloaf and mashed potatoes a sabotage might be to force another contestant to make mashed potatoes with potato peels or to make meatloaf with unground meat. When losers are eliminated they return their prize money to the host whereas the winner keeps whatever they didn't spend over the course of the show.
As someone who cooks frequently the show has no educational value. The techniques or tricks the chefs use aren't discussed. Instead the focus is put on how a chef deals with the obstacles heaped upon him. While amusing, it's unlikely your average viewer will ever need to cook a pancake on top of a tin can. Thus, unless your cookware is stolen by the international culinary mafia you probably won't learn much.
As an economist however, the auctions are fascinating. It's rare to see so blatantly how bad we are at making financial decisions under pressure. Here is a fictional, but representative, example of how cutthroat kitchen often goes. Remember each contestant has $25,000 to spend in auctions. Any amount they don't spend will be their prize if they win.
Allen, Beth, Candice and Dave arrive at the set. They're told they'll be cooking a pizza. As usual they have thirty minutes to complete the task after they grab their ingredients from the shows pantry. Everyone scrambles to grab the items they'll need to make a crust and sauce as well as some cheese and toppings to cover the pizza with.
Shortly after cooking begins the host starts to auction off sabotages. The first auction (of an expected four) is for the ability to force another contestant to make their sauce out of unripe tomatoes. Beth and Dave are both terrified of having to use unripe tomatoes and so bid against each other until Beth wins the auction for $5500. She awards the tomatoes to Candice.
Auction two is won by Dave who forces Allen to have to grate his own cheese from a block rather than using preshredded. Auction three is won again by Dave who forces Beth to choose from some common ice cream toppings instead of pizza toppings for his pizza. Finally auction four is won by Candice who gets revenge on Beth by making her cook her pizza on a griddle instead of in an oven.
The end result is Allen has to grate his cheese from a block. Beth has to use ice cream toppings instead of pizza toppings and cook her pizza on a griddle. Candice has to use unripe tomatoes in her sauce and Dave is unsabotaged. Additionally, Allen has spent nothing, Beth is down $5500, Candice has spent $5000, and Dave has spent $10,000. Judging commences and Candice is eliminated as the judges felt her pizza was the least delicious.
Round two commences and Dave is eliminated. Allen and Beth remain with $22,000 and $18,000 respectively. After the final round Beth is crowned the victor with winnings of $10,500.
Using this fictitious series of events we can point to a number of poor decisions contestants made along the way. In the first round Beth and Dave were willing to spend a great deal in order to avoid getting the under ripe tomato sabotage. This is a very common behavior as expressed by contestants. They often are willing to bid on an auction just to guarantee they are not stuck with the sabotage. However, in the first round losing an auction generally only costs a contestant a one third chance of being hit with the penalty (as the winner could award it to any of the three other contestants). Given this probability the prices contestants pay is borderline absurd. Contestants routinely bid as if losing an auction is a guarantee of receiving a penalty, a belief which is clearly incorrect.
Another oddity is that contestants are often most willing to spend during the first round of the contest. Bids of $4000 to $6000 are not uncommon in the first round while in rounds two and three items will routinely sell for under $3000. This is partially attributable to the fact that contestants that are loose with their money quickly deplete it or are eliminated thus leaving only more reluctant bidders with the assets needed for such lofty bids. Still, it's difficult to understand why these big spenders expend so many resources so early, thus leaving themselves vulnerable in later rounds where they're more likely to be hit with sabotages. A possible, if unlikely, explanation is they're attempting to put themselves in a position of weakness so that other players will award sabotages to stronger opponents. However, putting oneself in a position of difficulty in order to avoid difficulty seems rather nonsensical.
The most bizarre behavior is when contestants spread penalties relatively evenly among the other contestants. Each round one person is eliminated. The logical behavior is as soon as a person receives a sabotage all other sabotages should be heaped upon them. This eliminates animosity in later rounds (as the contestant will have been eliminated) and is the best method to ensure continuation through the rounds.
It would seem there are two possibly strong methods of bidding intelligently on the show given logical behavior (which the contestants never seem to exhibit). Either win the first bid or do not bid at all. If you choose to win the first bid you then award it to another player who then is signaled as weak to the remaining players. If that player does not win the second auction it should logically be awarded to them as well and they are now in a position where they are at best as weak as the next weakest player if they win the next two auctions and award them both to the same person. Thus the sabotaged player is forced into winning the second auction or having a very high probability of elimination. This method is aggressive and potentially resource intensive if played against another player with the same strategy. However, when facing three passive players (as described below) the resource cost is low. The considerable downside is whoever you award the sabotage to essentially must bid against you in the second auction or risk the piled on sabotages. If they win the second auction they're most likely awarding their sabotage to the person who sabotaged them in the first auction (you). So while you may gain a cheap sabotage to use against someone else, you also likely gain an expensive sabotage used on you. Overall it is a gain against the player you sabotaged, but a loss against the two players who are uninvolved.
The second, and likely preferable strategy is far simpler and likely superior. Do not bid on the first auction. The winner only has a one third chance of awarding the sabotage to you. Two thirds of the time you will be able to keep all your money and receive no sabotage for the first round (as all sabotages should go to the first person sabotaged or very likely the person who sabotaged them). In the unfortunate event that you are chosen for the first sabotage you are put in a situation where you must make another player as weak as you, meaning you must start bidding and winning auctions to weaken them. However, with the exception of a likely higher cost of auctions two, three and possibly four in the first round as compared to auction one you are at no particular disadvantage against the player you choose to target. Overall your expected earnings increase a great deal as well as your position in later rounds of the contest.
It would be interesting to conduct interviews with contestants in order to establish their motivations for how they award sabotages. Certainly the tendency to spread the penalties around seems to indicate contestants think more of equity than how best to improve their odds of victory. Still, at times I wonder if at times their consideration amounts to no more than, "I don't want to be stuck with that. I'll buy it then give it away to someone more or less at random." Regardless of motivation, the show is an excellent demonstration of how not to think about strategic decisions.
That's it for this week. Until next time stay safe and rationale.
Friday, January 3, 2014
The 25%
As Americans we are constantly reminded how fortunate we are that we're allowed to keep most of the money we earn. After all, in many nations the public toils all year long and the greedy government takes the fruit of their labor to pay for wasteful social programs.
The reality of course is a bit less extreme. The United States is near the middle of the pack when it comes to taxation as measured as a percentage of GDP (26%). Many Middle Eastern and Asian countries tax far less (Saudi Arabia 5%, Hong Kong China 13%) while many European nations tax more (Sweden 46%, Italy 43%). Thus in terms of keeping the results of our efforts we're neither terribly fortunate nor victimized.
The upper crust of France however will be finding their baguettes quite a bit smaller for the near future. Last Sunday the French high court upheld a 75% marginal tax on salaries over one million Euros ($1.3 million). The tax is largely symbolic as it is planned to last only two years and generate a relatively meager amount of revenue. However, it certainly sends a powerful message to the public, "There is no limit to what the government can take from you." Much like the Cyprus deposit confiscations this move can only terrify the wealthy while solving precious little.
I do not mean to give the impression that I believe taxes or governments are evil. Nothing could be further from the truth. Taxes are a necessary component of modern society. Further, this new French tax is hardly going to cause any millionaires to starve or even prevent them from upgrading their yacht next year (their first million euros are taxed normally). But, this move is a classic example of political machinations that will have real detrimental economic impact.
French president Francois Hollande himself has said that the tax is symbolic and unlikely to substantially impact France's problematic revenue shortfalls. Why then institute a tax which will undoubtedly drive many wealthy French from the country, thus removing their tax revenue from the system entirely? In the long term it's likely that tax revenue will decrease as once the tax expires two years hence those that emigrate now will still be gone. Not to mention the extra reluctance of any wealthy considering moving into France.
What is gained by such a symbolic maneuver? Hollande claims it's about economic fairness. However, I doubt many people would claim that confiscating 75% of anyone's wages is particularly fair, especially if they receive nothing more than those who are taxed far less. More likely this act of political theater will simply increase tensions between the haves and the have-nots while doing nothing to solve France's fiscal issues.
It makes sense that the wealthy should bear more of the burden of taxation. Thus the logic of a progressive tax system such as we enjoy in the United States. But to tax any person for three quarters of their wage, even if it is only for Euros earned over one million, is ridiculous. To do so when it does nothing to solve revenue shortfalls is bordering on criminal.
That's all for this week. Until next time stay safe and rationale (and probably out of France).
The reality of course is a bit less extreme. The United States is near the middle of the pack when it comes to taxation as measured as a percentage of GDP (26%). Many Middle Eastern and Asian countries tax far less (Saudi Arabia 5%, Hong Kong China 13%) while many European nations tax more (Sweden 46%, Italy 43%). Thus in terms of keeping the results of our efforts we're neither terribly fortunate nor victimized.
The upper crust of France however will be finding their baguettes quite a bit smaller for the near future. Last Sunday the French high court upheld a 75% marginal tax on salaries over one million Euros ($1.3 million). The tax is largely symbolic as it is planned to last only two years and generate a relatively meager amount of revenue. However, it certainly sends a powerful message to the public, "There is no limit to what the government can take from you." Much like the Cyprus deposit confiscations this move can only terrify the wealthy while solving precious little.
I do not mean to give the impression that I believe taxes or governments are evil. Nothing could be further from the truth. Taxes are a necessary component of modern society. Further, this new French tax is hardly going to cause any millionaires to starve or even prevent them from upgrading their yacht next year (their first million euros are taxed normally). But, this move is a classic example of political machinations that will have real detrimental economic impact.
French president Francois Hollande himself has said that the tax is symbolic and unlikely to substantially impact France's problematic revenue shortfalls. Why then institute a tax which will undoubtedly drive many wealthy French from the country, thus removing their tax revenue from the system entirely? In the long term it's likely that tax revenue will decrease as once the tax expires two years hence those that emigrate now will still be gone. Not to mention the extra reluctance of any wealthy considering moving into France.
What is gained by such a symbolic maneuver? Hollande claims it's about economic fairness. However, I doubt many people would claim that confiscating 75% of anyone's wages is particularly fair, especially if they receive nothing more than those who are taxed far less. More likely this act of political theater will simply increase tensions between the haves and the have-nots while doing nothing to solve France's fiscal issues.
It makes sense that the wealthy should bear more of the burden of taxation. Thus the logic of a progressive tax system such as we enjoy in the United States. But to tax any person for three quarters of their wage, even if it is only for Euros earned over one million, is ridiculous. To do so when it does nothing to solve revenue shortfalls is bordering on criminal.
That's all for this week. Until next time stay safe and rationale (and probably out of France).
Subscribe to:
Posts (Atom)