The Reality of Pareto

I was in a training session at work for a new quality program that was being implemented.  They flew this guy over from the states who was the global training lead for this new program.  He asked the class if anyone knew who Pareto was.  I said he was an economist.  He asked me to provide details.  I provided a quick summary of Pareto's work (after all, we did spend an entire class one school day in Economic Analysis of the Law discussing Pareto efficiency).

Trainer:  Um... well, that's not the same guy as who I'm thinking.  Basically, Pareto was this guy who had discovered a very important statistical trend that is today known as the Pareto Principle.  Blah, blah, blah...

Me:  Ugh...  /wanted to desperately explain to the guy how they're exactly the same person.

OK, quick note on teaching.  When you are teaching or training people, there will always be someone who plays "stump the prof".  I do this now and then.  I see lots of people do it.  When we do this, we're not trying to make the "prof" look bad.  We're genuinely curious about something and want it figured out through dialogue.  Or we're bored with the discussion and want to turn it into something more enlightening.  Or we're annoyed that something incorrect is being taught.  Or maybe sometimes we're just mean, but I don't think there are many who really care to do that.  They have other motives, being students.  If you're teaching and you encounter a "stump the prof" situation, don't feel embarrassed.  Don't back off.  Don't get combative.  Engage with your students.

OK, back on topic.  The Pareto principle basically describes how the majority of stuff is associated with the minority of people, causes, etc.  Normally, the numbers are 80-20: 80% of the wealth in a nation can be owned by 20% of the population; 80% of the problems are caused by 20% of the causes.  Etcetera.  We were being trained on how to identify the 20%.  Basic stuff that you learn in any business school.

When you look at movements like Occupy Wall Street, they've altered the framework a bit.  Now it's the 99% vs the 1%.  It's not like the 1% hold 99% of the wealth, but they hold a significant amount (about 40%).  Unfortunately, due to various factors, the 1% is probably a systemic, natural result.  Even if the 1% are broken down through revolution, a new 1% will just arise.  This seems to be a natural power law.

Case in point?  Look at China.  There was a lot of strife because of, among other things, the big divide between the rich and poor.  So Mao rose up and started the revolution, with the peasants landing on top.  Well, over time, we have arrived at the exact same situation again.  The wealth gap between the 1% and 99% is way bigger than what exists in the USA and is much more visible.  This is probably a simple side effect of the fact that China's population is much bigger, so percentage calculations result in amplified gross numbers.  It is very odd to hear people in the west constantly talking about how rich China is now, how China is the benefactor of the US due to being owed so much US debt, etc.  The fact is that these statements may be true at a macro level, but at a micro level inside the country, it's not such a simple picture.

There is a HUGE wealth gap between the rich and poor in China.  You have millionaires who have no problem buying tons of fancy cars, expensive brand name bags, bubble-level real estate, etc.  Then you have the girl in the bakery who's making only $150 per month, while the most basic apartment also costs $150 per month.  Lucky for most employees, the employer is usually willing to provide living quarters at a cheap price.  Of course, it usually comes out of the $150 per month, unless it's in the factory where it might be free.  Well, in the factory, you make more money, but it's a dog fight to get in, and the conditions are horrible by western standards.  Even further, there are still many who live in villages without basic plumbing.  Since the wealth gap is many times blamed on corruption, the general populace is very upset to the point where any rich girl flaunting pictures of her wealth on the web causes a national media circus (and tough questions for parents that might work for the government).  The complaints are very similar to the ones that Occupy Wall Street make about Wall Streeters and the bailouts in the US.

So we have this strange dichotomy where the west views China as very rich, with growing power, while inside China, the poor are plentiful.  This dichotomy is not made any easier from the fact that it was the peasants who took over and then sent any remaining rich/powerful/freethinking people to re-education camps.  Basically, the whole country became poor, and all the rich/powerful/freethinking people who had the means left for locales like Taiwan, HK, etc, before things got really bad.  And if you're a believer of what Malcolm Gladwell wrote in Outliers, it's hard for a people group to shake attitudes and lifestyles that have been in place for hundreds of years, meaning that it's difficult for many of these people to stop thinking like peasants.  And yet, a new 1% still arose.

Think about it.  We have a real-world experiment that turned an entire large country upside down in the quest to eliminate wealth gaps, but failed in that quest.  The corresponding interesting question is whether any other ideas would really solve the issue.  It's easy to say that China failed to solve wealth gaps because Communism doesn't work as an economic system (which is an entirely different subject altogether and will not be discussed here).  The thing is, capitalism hasn't really solved the problem yet either, if it will ever.  Very few countries in the world actually have solved it.  The exceptions are a small number of European nations, but the difficulty scale of their problems is much smaller due to the smaller population sizes.  Never mind that they have their own economic issues right now.

If anything, I think the symptoms of the Pareto principle will be more entrenched over time.  There are a number of factors that I believe will feed into this, but I think the primary ones to consider are globalization and technology.  I think these two factors do more to destroy jobs than anything else.  But don't label me as a left-wing free market hater.  I also think they do more to create jobs than anything else.  The problem lies in how to cross the chaotic chasm in between point A (jobs destroyed) and point B (jobs created).

Globalization is the easy one to analyze.  If you open up international trade so that anything can be manufactured and sold anywhere, simple product manufacturing (eg. clothing, accessories, trinkets, etc) will always be manufactured wherever the labour is the cheapest because not a lot of capital investment is required (relatively speaking).  So the jobs for manufacturing simple things will disappear in the wealthier nations and become abundant in the poorer nations.  We saw this happen with simple manufacturing moving from western nations to Asian nations like Korea and China.  Now it is happening again, moving from China to Vietnam, Thailand, Indonesia, etc.  Imagine that.  China is losing jobs because of globalization.

Once again, the important question?  In the wealthier nations, who are losing those jobs?  Well, those jobs are usually filled with unskilled labour, so therefore cheap labour.  It's the uneducated, unskilled, and poorly paid people who are losing the jobs.  And because they're uneducated, unskilled, and poorly paid, it's obviously difficult for them to find new jobs, once the jobs they were capable of doing get moved overseas.  The effect is exacerbated when even office jobs like tech support, legal support work, etc, get shipped overseas due to outsourcing.  So the rich will stay rich, and the poor will stay poor.

It is interesting to note that for high-tech manufacturing, many people in the industry agree that cost of labour does not factor primarily into the decision.  Read interesting articles on that here.  And here.  And here.  And finally here.  I remember hearing a talk from the CEO of a new solar panel company based in Vancouver, BC.  He was asked by an audience member why the company chose to build a manufacturing plant in the BC interior instead of in China.  He said it was because he was proud to be making a Canadian company and the cost really isn't that different.  The main cost is in the capital investment, plant equipment, etc.  That cost is similar whether in Canada or in China.  The labour as a percentage of cost is tiny and therefore doesn't factor into the decision.

Technology is a bit more out of left field.  Technology is supposed to free our minds and give us new abilities, or at least that's what the advocates say.  And to a certain extent, that's true.  It is the case that people in the middle east were actually naming their newborn babies Facebook to celebrate how social media and the web had played a part in coordinating the Arab Spring Uprisings.  It is the case that there was no way you could make money as an SEO expert 20 years ago (the concept didn't even exist because the web didn't really exist).  But it is also the case that the primary purpose of technology in a company is to increase productivity.  And the most natural way for it to increase productivity (perhaps the only way) is to automate tasks.

This hit home for me only recently, as I reflected upon my career.  I started out as a software developer for a telecom company in Canada.  We were a special swat team that went throughout the organization to automate solutions for difficult or stupid business problems.  All of us were recent grads from university.  So you had a bunch of bright-eyed kids automating systems and processes being handled by people who had worked at a job for 20 years.  All of a sudden, no job.  Said people were either given a package for early retirement or reassigned to new work.  Except within a few years, maybe that new work would get automated too.

Technology has a huge ability to create new opportunities.  But in the process, it destroys the old ways.  It is a natural economic phenomenon labeled by Joseph Schumpeter as creative destruction (well, according to Wikipedia, maybe he adapted the idea from the Marxists).  The problem exists when the people who were depending on the old ways for work cannot easily retrain themselves for the new economic paradigm.  And when you have a ton of people who are unable to easily reinvent themselves, you have a lot of unemployment.  I remember discussing things in a PMP prep class with some fellow classmates.  One lady had commented on how her company implemented SAP and then was able to lay off the entire accounting department.  This technology stuff is scarily powerful.  P&G is laying off thousands from its marketing departments after discovering how much cheaper it is to just market stuff on the web with some web marketing specialists (greater economies of scale for one thing).

The important question this time: why can't these people reinvent themselves easily?  First off, reinventing oneself is not an overnight thing.  People go to school for years to attain the skills they have for a job.  Or they're an apprentice for some time for some trade.  Either way, you're looking at a significant time investment and income drop just to get your foot in the door.  And employers don't want rookies in various fields where they have need.  No, employers want qualified people that can hit the ground running.  It costs too much to take inexperienced people and train them up.  Especially when those inexperienced people are demanding high wages because they still have a mortgage and kids to feed.  It's only after reading about the stories of the long-term unemployed that I realize there's a huge disconnect here between what technology hopes to do and how newly unnecessary employees are supposed to become employable again.  There are jobs out there right now.  High-quality software engineers simply cannot be found in a very big tech boom.  Again, SEO experts didn't even exist 20 years ago.

What's the answer?  I don't know.  There are much smarter people than I working on these issues.  That being said, I think we can conclude that the most important skills people need these days and well into the future are not in specific technical skills or domain knowledge.  That comes and goes and may get totally automated tomorrow.  Traders?  Probably.  Doctors?  Possibly.  Software developers?  Heck, I saw prototypes of stuff that automated simple application development back in 2005.  The list goes on.  Yeah, some small niche of jobs in whatever category has always remained after getting totally automated.  But that doesn't comfort the vast majority of people in those job categories who had to reinvent themselves.

Rather, the most important skills are probably the ability to learn and adapt to new environments quickly, think on one's feet, be creative and innovative, and take calculated risks to do new things.  These are skills that people already need to learn today.  But they'll probably be even more important in the future for that time when your job is suddenly outsourced to another country or automated out from under your feet.  Technology's capability to change the world seems to be only accelerating, so it'll be harder and harder to adapt in a timely manner.  Good luck, huh?

The last thing I'll say is that I think it's unfortunate when people think they can solve the negative effects of globalization and technology through policy.  If the two things are unstoppable forces, no policy is going to prevent them from doing their damage.  The policies that need to be created need to focus on containing the damage and prepping for the new reality so that the pain endured during the transition is minimal and everyone is ready to party rock when the transition is complete.  That would probably involve a lot of education and training programs, but hopefully not for skills that will become obsolete quickly.

Notes on The Karate Kid Remake

1.  This movie was a lot better than I thought it would be.  It almost makes me forgive them for not calling it The Kung Fu Kid.  Almost.

2.  The athleticism shown in this movie is way better than the original.  But are children's kung fu tournaments in Beijing really that violent?  Seriously, some of them look like preteens.  Not one looks a day over 13.

3.  I understand why Daniel looks so small in the original Karate Kid compared to the bullies.  But this remake seems to overdo it a tad.  Dude, she looks WAY older than him.

4.  This movie did not provide a good representation of DDR.  But girl can move.

5.  I remember watching the Mighty Ducks movies and thinking that I've never seen midget hockey get so much media in real life.  I wonder if these children's kung fu tournaments actually get that kind of rock star treatment in real life.

6.  I think any normal kid would have gotten tired of the jacket thing after two days, max.

7.  Pretty faithful to the original story, but nice twist at the end on switching masters.  You can tell there won't be a Karate Kid 3 remake anyway.  :)

8.  Do people really do that with their chopsticks in China and continue eating?  I know some people think they're uncivilized and all, but I've never seen it happen. For that matter, I haven't see anyone drink water straight out of the tap without boiling it either....  betcha Jackie Chan's water was filtered and boiled for that scene.

9.  Was the mom hitting on Jackie Chan?

10.  When did he have time to practice the cobra hypnosis?

Best description I've ever seen on why many big corporations have difficulty improving operations

Yup.  This is exactly what it's like, folks.  Don't work for one if you don't want the pain.

Let me tell you why you don't see this happening more often. 

I did this on a project a few years back. I replaced a paper workflow process that was taking up two people each in three departments with a web-based workflow that increased visibility, dropped turn-around time from days to minutes, increased accountability and accuracy and trimmed those 16 person hours of processing down to 1-2 per department. 

Everyone who directly interacted with the new system loved it. Numerous edge cases that would have been lost in high-level review were caught and integrated from day 1 due to my actually watching people do the job for a day or two per department. The solution has been rock solid (minor maintenance only) for five years. 

And I almost lost the job.

The people who sign the checks were furious. The balance of political power between departments were thrown for a loop. One head in particular treated the thing as a near-existential threat. His entire concept of his job revolved around being the authoritative interface for retrieving and maintaining pieces of data that were no longer exclusively under his control. Another flipped out because middle management saw the results as cause to reduce his headcount and budget, and thus importance. 

These two departments fought for months, refusing to contribute their shares of budget that were pledged toward modernizing this system. 

On a technical and practical level, it was the single best experience I've ever had as a consultant. On a personal and economic level, is was one of the worst. It was some of the hardest money I've ever tried to collect. It was some of the most time and energy I've put into the political and 'sales' side of a job (the part I treat as a necessary evil, but very much evil). The corporation has made out like a bandit in the long run. But I paid the price. 

It's simply too easy and financially rewarding to allow a client's political nonsense to screw up every stage of a project. I have less stress, the people who pay me are happier and I bill far more hours. 

As with most software, internally developed software included, you don't see better projects more often because the incentives are horribly perverted and stacked against it.

Who owns the data?

I haven't blogged in a long time.  Various items I want to blog about are in various forms of draft.  But they're all long posts.  So here's a quick one.

It seems that the widow of someone recently deceased would like her late husband's profile removed from Facebook.  Wow, that's complicated.  Me, not being a lawyer, can already see huge problems here if Facebook were to comply with her request.

Facebook is no stranger to privacy kerfuffles.  Note that what I say here is neither for or against how Facebook handles privacy, it is simply stating what is reality. With that in mind, let's look at something that Zuckerberg posted back in February 2009:

Our philosophy is that people own their information and control who they share it with. When a person shares information on Facebook, they first need to grant Facebook a license to use that information so that we can show it to the other people they've asked us to share it with. Without this license, we couldn't help people share that information.

One of the questions about our new terms of use is whether Facebook can use this information forever. When a person shares something like a message with a friend, two copies of that information are created—one in the person's sent messages box and the other in their friend's inbox. Even if the person deactivates their account, their friend still has a copy of that message. We think this is the right way for Facebook to work, and it is consistent with how other services like email work. One of the reasons we updated our terms was to make this more clear.

In reality, we wouldn't share your information in a way you wouldn't want. The trust you place in us as a safe place to share information is the most important part of what makes Facebook work. Our goal is to build great products and to communicate clearly to help people share more information in this trusted environment.

Facebook followed through with this philosophy by implementing various privacy features and levels.  More interestingly, they allowed you to download and export all of your data to your harddrive.  If you're able to take all of your data from out of Facebook and download it to your own repository, I think that's a clear statement on their view of who owns the data.  You own it.  Of course, another question arises as to whether it's actually useful or relevant outside of Facebook, but for this discussion that's irrelevant (well, if you're a young legal genius, you might grab some press).

The key point is that if you own the data, then you decide what happens to it.  Other people can't.  So what happens to other property owned by you when you die?  Well, if you have a well-written will, it's clear.  If you don't have a well-written will... well, I'm not an expert on this at all, but some initial reading makes this subject sound pretty complicated and potentially the area of many ugly lawsuits, depending on what property's at stake.  For the sake deciding who gets to own a deceased person's Facebook data, would such ugly lawsuits come into play?  Who knows, some people are strange.

The point is that you own your data, and if you own your data, it's not so obvious to pass own ownership to a specific person.  If that specific person does not have ownership of the data, then how can that person demand that Facebook delete it?  Any holes in my logic here?

Now, let's look at the potential pitfalls of Facebook setting a precedent of complying with this lady's request.

1.  They open up a huge gateway for fraudulent/malicious activity.  Imagine a jealous sibling who always hated the deceased.  This person wants to erase any memory of the deceased.  Would be easy enough to even get all the valid documentation and submit it to Facebook to get someone deleted.  Now imagine it happening by people who aren't even related in any way.  There's lots of documentation these days about social engineering on Facebook to get fake friends for nefarious purposes.  Opening the door would allow all of that malevolent force to do some really nasty stuff.  Ever seen what high school bullies can do to fellow classmates?  To kids who committed suicide because of bullying?  Etc.

2.  You can't delete 100% of the stuff anyway without going against Facebook's own policy (see above).  Facebook made the decision to model their policy based on how the broader Internet works.  So the question is now then is the concern only about Facebook?  If so, why only Facebook?  Why not other channels where information is shared?  Other social networks like Google+, LinkedIn, Twitter?  E-mail?  Online filesharing services?  If you single out Facebook without attacking the others, I think Facebook would have very good legal ground to fend off your attacks.  If you want to include everyone and change up the entire Internet... well, that's not going to happen easily.  It would require rearchitecting decades of Internet technology and policy worldwide for one thing.

3.  What if Facebook ends up becoming legally responsible to retain data for evidence purposes in the future?  E-mail has become such a ubiquitous utility for communication that many companies and e-mail providers are required to submit data to authorities when requested.  Of course, depending on individual corporate policies, they'll also decide whether such information requests are legal and valid.  But the point is, a process exists for this to happen.  In telecommunications too.  Facebook aims to become a global communication utility, that's been Zuckerberg's aim from the beginning.  If they truly achieve that goal (some argue they already have), is it a stretch that Facebook would also have to comply with these types of information requests on a legal basis?  Actually, aren't they already?  And if so, wouldn't allowing 3rd parties to delete someone's data instead of only the owner of that data cause some difficulty in this matter?  What if the deceased were an assassinated mob boss who the police are now investigating?  Would investigations be hampered by Facebook complying with 3rd party requests to delete the data?

4.  What if the guy never wanted his stuff deleted?  He never made it clear.  This is like a digital version of the euthanasia debate.  If he never wanted his stuff deleted, but we can't ask him anymore whether or not he wants his stuff deleted (or if he even cares), then how can we easily decide whether or not we should delete his stuff.  It's obviously not as controversial as euthanasia, but the logical/ethical conundrum is similar.  It's a tricky road to walk, and like any tricky road, decisions shouldn't be made without heavy analysis and debate first; otherwise, you end up with drastically negative unintended consequences.

I understand that this lady may feel anguish whenever she sees this guy's photos, etc.  I'm sorry if this sounds callous, but that also seems a bit immature.  We have photos of loved ones so that we can remember and celebrate their lives.  If this were the 80s, would this woman take all her photo albums and throw them in the trash?  Maybe.  OK, her decision.  But those photos would probably be owned by her clearly, and there would probably be no large ramifications if she threw them out.

Here, we have someone who is unable to deal with her anguish and wishes the pain to disappear by getting Facebook to do something that may not be very good for anybody.  My thoughts are that she should realize that it's precisely because they loved each other that she should keep those photos on Facebook.  That's also a better resolution for everyone else who's on Facebook.  Facebook is constantly pushing the boundaries on what entails privacy, for better or worse.  Remember, I'm not commenting on whether or not I think Facebook's policies are good.  I think there's some good and some bad.  But I hope that nobody thinks poorly of Facebook for turning this lady down.  It would not be a good precedent.

The biggest lesson from here?  Looks like we all need to specify in our wills now who owns our Facebook data.  Then what about my gmail?  Chee, complicated.  @@

Another day in the life

So I went back to the gym yesterday.  Hadn't gone since the Body Pump class.  Wanted to try something different, so I joined the Body Combat class.  No idea what to expect.  It was aerobics.  Somewhere near the end, there were some push-ups involved.  The instructure was not a TCG (tiny chinese girl), but even so, I've never seen a girl out-push-up a guy easily.  I'm not saying I'm a marine, but I can do a few.

But when these push-ups came, I was so exhausted, I couldn't even do push-ups with my knees on the ground.  These classes are freaking embarrassing.  And all the time, the girl's yelling and grinning.  "Jia you!  Come on!  OK, look at me, follow me!  Knee!  Knee!  Hands up!"  Gah.

Later, I was bending down to wash my ankles in the shower.  My forehead hit a shelf.  It didn't hit the shelf hard, but I felt the shock all the way along my neck, right shoulder, right arm, and fingertips.  Oh, pain.  That'll hurt.  Normally, when you bump your body, your body is able to absorb the shock easily, right?  Not this body.  It was so worn out that even the tiniest shock caused horrible damage.  I decided to go for a massage after because I just knew it would be too painful the next day.  It's traditional Chinese treatment, where they even do this cupping thing.  They put these glass cups on your back, and use a dash of fire to suck out all the oxygen.  It creates a vacuum that sucks up your tissue and blood and is supposed to make things better.  I have no idea if it works.  I now have all these dark spots all over my back and shoulders.  But I am glad I got the massage.  I am able to move around today, unlike last week after Body Pump, when I could barely move.

Seriously, why?

A day in the life

Yesterday, I went to the gym for the first time in a long time.  It's a different gym than before.  I got one of those passes that have a limited number of uses.  The front desk girl told me that the Body Pump class would be starting at 7:40pm.  She said check it out.  OK.

I went and did some treadmill for 10 minutes, and then went to the room where they were going to do Body Pump.  Everyone was laying out equipment on the floor.  Low platforms and barbells?  OK, I can do this.  The class trainer is this TCG (Tiny Chinese Girl).  Yeah, she looks really fit, but she's only a TCG, right?

I never thought a TCG could outlift me.  I am in so much pain today.  For the entire hour, I could barely keep up with her.  My legs felt like collapsing.  And she just kept on going with this big grin on her face.  "Jia you!  Come on!  YES!  One more!  OK, go!  Houmian de nansheng, ni keyi ma??  Jia you!!"  Nod yes, I'm OK, with gritted teeth.  And now today, can barely move.

Went to Mandarin class today after work.  My teacher signed me up for this foreigner speech competition.  She's been bugging me about it for 2 or 3 weeks now.  I keep telling her to find someone else, because my Mandarin is still pretty bad.  But she says no, my Mandarin is by far better than all her other students.  And there's an 8000 RMB prize.  OK, OK, fine.

What do I find out today?  It's a multi-round event and somehow I'm already in the semi-finals, even though I never participated yet.  Furthermore, I have to not only give a speech, but also sing a Chinese song (or do some other kind of Chinese cultural performance).  Maybe I can come down with a big case of diarrhea Friday night.  The semi-finals are on Saturday.  I have to learn a Chinese song in 2 days?  :(

A day in the life.