Using Sentiment Analysis To Improve the Luongo App

Oh not, not another goal let in by Luongo

So last night, Benji and I really stepped the Luongo app game up to another level by adding sentiment analysis in deciding what people were thinking.  Before, our main algorithm was really elementary and only detected people who tweeted the words “luongo” and “fault” and then counted all the tweets that had the word “not”.  We would then compare the number of tweets that had the word “not” (indicating that people didn’t think it was his fault) with the number of words without the word “not” (indicating that people did think it was his fault).

Here’s a snippet of the code so you see exactly what it’s doing:

results = tweepy.Cursor(api.search,
q="luongo & fault",
rpp=100,
result_type="recent",
include_entities=True,
lang="en").items()
for r in results:
#print r.text, r.id
# r now contains one tweepy result object
if r.text.find(u'not') > -1:
n += 1
else:
nn += 1
count += 1
if n > nn:
print "That was luongo's fault"
else:
print "That was not luongo's fault"

Pretty basic.

Now, what we’ve done is we actually take the data that we get from the Twitter API and send it to another API called Sentiment 140 to do sentiment analysis on all of the tweets featuring the word Luongo.  Sentiment 140 will analyse each individual tweet that says if it’s a positive, negative or neutral tweet and then we count up that value like before.  The reason that this is better is because Sentiment 140 will actually look all words within the tweet which contain positive or negative values instead of just looking for only the word “fault” as a subject and “not” as the counter indicator.  For example, the following is a snippet from the JSON response from Sentiment 140.  It’s ranked as a negative tweet (negative tweet has polarity = 0):

Sorry we broke your goalie, mister. #Luongo #Canucks Go #Bruins”,”polarity”:0

Here’s a tweet that’s ranked as positive (postive tweets has polarity =  4:

@loveforthree Pretty good for a team that needs Luongo…”,”polarity”:4

So asides from showing that Bruins fans are terrible, Sentiment 140 does an excellent job at figuring out whether a tweet is positive or negative, especially compared to our previous algorithm.  There’s a couple of challenges that we’re facing, which is whether we’re only reporting on tweets with the word “fault” in them, or just assume that any tweets that are positive and negative during a game will generally have some bearing on the number of goals scored.

Our next step is going to be importing this script to a server so you can all start accessing it during games!  It’s gonna be cool.

 

Geeking out with Python – Was That Goal Luongo’s Fault?

Was that goal luongo's fault?

As some of you may know, I’ve been taking Computer Science 101 on Udacity, which is an online university which I think is fucking awesome.  Their structure of 1-5 minute youtube videos with quizzes shoved in the middle totally works for me compared to stuff like Coursera and MIT Open University.  But hey, what doesn’t work for me might work for you and vice versa.

Anyway, I’ve been relearning python (which I took in “real university”) and it’s been pretty interesting.  Udacity’s first unit is actually way more difficult than the entirety of my CS class, which I found pretty surprising.  The main thing that Udacity is really clear about is that programs are really just problem solving broken down into very basic steps.  That’s probably worth the price of admission alone (which is $0.00).

Late one night while watching a Canucks game, I found myself having a problem that I always seem to have which is whether the goal that was let in Roberto Luongo was actually his fault, or if my bias was blinding me.  Then it hit me.  What if we could create an app that would go on twitter and tell me whether people online thought it was his fault?  And that’s exactly what we did.  After signing up for the Twitter API key, we got to work hammering on the Tweepy module and started to build code on previously documented code.  It only took about an hour to build the first alpha version of the code, but holy shit it was the most exhilarating thing I’ve ever done on a computer.

We’re going to improve the algorithm that calculates the mood of twitter, but for the time being the code has spoken:

result of luongo app

 

Diving in face first with LAMP – Part 4: I fucking did it

First look at my LAMP Blog

From what I hear, men do not have the ability to give birth so I’ll just have to assume that this is what it feels like.  After a long ass time of fiddling with stuff, I have finally created my first blog on a LAMP stack.  Disclaimer: I did not write the PHP myself, I just copied and pasted it from this Cutting Edge Tech (which was an awesome guide!).

So why was this project important? Well, asides from proving to me that dreams can come true, building this blog really taught me how front-end development and back-end development come together to make the websites that we know and love.

It’s a lot like when I first started watching skateboard videos.

When I watched my first skate video, my mind was blown in a way that could never be unblown again.  Up to that point, my skateboarding experience consisted of rolling outside of my friend Daniel’s house or the skateboard park at the local park.  The Tiltmode Army video (that I got from Westbeach on Boxing Day) was the very first time that I saw “street skating”, which was people doing tricks on the street.  From that day forward, I never saw architecture the same again.

That set of stairs? Perfect for a 360 flip.  That rail? Dying for a kickflip to crooked grind.  Everything on the street became a canvas for what trick could be done on it.  Even to this day, I’ll see a big ass handrail on the street and thing of what awesome trick I would love to see be done on it.  I think that building a blog with a LAMP stack is the same thing.  I can see how things are made.  Wordpress?  Looks like PHP linked to MySQL with some nice CSS on top.  Facebook?  Looks sorta like PHP and MySQL but crazier.  I’m probably not entirely right, but at least now I have a glimpse into the interactions between the front-end and the back-end and how information is retrieved and stored.

I’m barely scratching the surface of what it takes to see a web app and see how that problem is being tackled, but I am starting to sort of get it.  It’s been a really rewarding experience to bang your head against a wall for so long and finally see something come of it.  I’m already googling other easy PHP/MySQL projects to do for the future and I hope I get even more skilled at how to do this stuff.

 

Why Startups Need To Think About User Experience

 

taxi

I was lucky enough to check out the DX3 digital marketing conference today and among the booths that were plugging the latest and greatest was the booth for the app, Hailo.  I’ve heard a lot about Hailo and the whole mobile transportation trend, which for me, is dominated by Uber.

Basically, Hailo is an app that allows you to hail a cab from one of their participating taxi drivers and it also allows you to pay for the ride using your credit card through the app.  Uber is similar, but instead of having taxis, they use town cars (think the blacked out sedans that the president rides on) or SUVs (Finally! We can fit more than 4 people in 1 car!) and it also allows you to pay via credit card using the app.

I asked the very nice person handing out coupons for Hailo rides to give me the pitch and I also told her how I’ve used Uber before.  The main selling point for her was that Hailo does not charge the premium that Uber charges, which is about 20% and the size of their fleet of cars was the 2nd largest in the city (vastly outnumbering Uber’s town cars and SUVs).

To me, Hailo’s thesis that people want a more convenient way to get a cab is fundamentally flawed.  For the most part, I’ve never had to call to get a taxi in my neighbourhood, or most of the areas that I go to.  Having the ability to make pay via credit card through the app is nice, but that’s not really the major pain point that people have.

The reason that Hailo will fail and Uber will win is because people fucking hate cabs.  I cannot personally recall a single time where I was pleasantly surprised by a cab.  I may have liked the driver, or it might have been cheaper than I expected to get from point A to point B, but for the most part the experience sucks.

Here are several reasons just off the top of my head:

  1. They don’t even try to hide the fact that they do not drive the fastest route.
  2. I have to ask if they take credit, which is a formality in the first place since all cabs are equipped with credit/debit machines (or an old school ticketing system which just scream “fraud”) and 50% of the time they get incredibly sketchy and say “Cash Only”.
  3. They do ghetto things such as ask for “flat rates” aka. off the clock prices which are usually above market price.
  4. Good luck if you live in the suburbs.

Whenever I use an Uber, things are vastly different.  The cars are unmarked and in immaculate condition.  The people that drive the cars seem to take pride in presentation of the vehicle.  Every single person who has taken an Uber with me is immediately infatuated with the service.  As a matter of fact, I have actually (drunkenly) taken an Uber to meet up with a lady instead of a cab because I wanted to appear more “baller” when I met her.

The major lesson from this is that that I think startups needs to take into consideration the product that they’re actually serving and not just adding features that help the distribution of the product.  In Seth Godin’s amazing Startup School podcast, he makes an amazing point, which to paraphrase, states that the price barrier for a product is 1 cent.  It’s either free or it’s not.  If it’s 1 dollar it might as well be 10 dollars and if it’s 10 dollars it might as well be 20.  As soon as the customer makes the choice to pay for the actual good, it really doesn’t matter how much money they give you.

I might be completely wrong with Hailo.  Maybe they put their participating cab drivers through rigorous testing and perform constant quality checks.  The problem is that I’ve already had a bunch of terrible experiences with cabs and they don’t seem like the best product to build a business on.

Hopefully that Hailo coupon I received at DX3 can pivot into a better product – in the recycling bin.

 

The Linkedin Scam?

Linkedin email premium screenshot

 

I don’t know about you, but I get emails from Linkedin every couple weeks trying to recruit me into their “premium” account service.  “You can be a featured applicant! How much would you love to be on the top of the HR manager’s hiring list?”  My first instinct is to say that any company that hires someone because they are on the top of their Linkedin mailbox is probably terrible and filled with people trying to buy their way to success.  It sounds insane to me that you have to actually pay in order to get attention from an HR manager.

When you look at all the major tech companies like your Facebooks and your Googles, all I hear is that they pile a significant amount of resources in recruiting the best kids from the best schools, poaching from the best companies, or even straight out buying out a company in order to get their team (Kevin Rose with Milk, for one).

It seems very odd for a company to try to get people to pay for a service who are probably unemployed or underemployed.  This reminds me a lot about a company called Keiretsu Forum that was charging entrepreneurs thousands of dollars and equity in their company in order to pitch to “investors”.  Luckily, thanks to Jason Calacanis, they were exposed and dropped their fees for early stage startups.

To me, Linkedin is slowly becoming the weird/gross pyramid scheme-y relative that’s trying to sell you steak knives or Tupperware.  Last month, my highly social and fashion blogger co-worker got an email from Linkedin stating that she had one of the top 5% most viewed profiles for Linkedin.  I was impressed and extremely jealous.  I quizzed her on what tactics she used in order to get so much traction to her page.  Did she have it on her business cards?  Was there a big ol’ button on her blog?  Unfortunately, she really didn’t know why she was so popular on Linkedin.

Later that day, I got this in my inbox:

you were in linkedin's top 10% emailSadly, here are the stats required to be in the top 10%:

linkedin views

All it took to make it to the top 10% of a social network with 200 million users was 72 views over the course of 3 months.  24 hits per month on average.  That’s not a lot of views.  Definitely not enough to write home about.  To be honest, when I received that email, I felt pretty disgusted with Linkedin.  Not only did they reduce the value of my Linkedin account, they reduced everyone else’s account.  People were tweeting and posting their score as if it was a validation of their professional ability but after finding out all it takes is 24 hits a month on your profile, it just became sad.

It’s troubling that a company that could have so many monetization models has to settle for the scummiest way to make money.  To be honest, I’m still going to have my Linkedin account, but there is absolutely no way in hell that I’m ever going to become a premium member.  A lot of young professionals really view Linkedin as the potential ticket to the big show, but I don’t think that any social network is really going to make or break someone’s ability to start or further develop their career.  I think the real key to success is to develop your skills further and to enjoy the learning process and not to fall for any pay-to-play traps like Linkedin premium services.

 

Raspberry Pi + XBMC = great hacking fun

Raspberry Pi

 

After months of hearing about this device from blogs and on reddit, I finally was able to get my hands on one of these devices from my coworker.  I spent the better part of yesterday trying to get Raspbmc, a linux distribution based on XBMC on my device and for something that cost $35 dollars, I’m tremendously impressed with how well it’s performing.  That’s not to say that there aren’t any flaws (you can’t really change the fact that the device is, and always will be slow) but it’s definitely a fun project for anyone that’s interested in stepping their linux game up.

 

Canucks game on SportsDevil

I was able to get everything up and running and was able to stream some of the canucks game using the SportsDevil add-on.

 

Highly recommended!

 

Diving in face first with LAMP – Part 3: Good is the enemy of great

That time Marge Simpson is in police training and climbs over a wall when there's a door.

After a couple hours of trying to manually install the LAMP stack, I’ve officially decided to give up on the manual install and just use TASKSEL to do it for me. TASKSEL is a command that will automatically do very common tasks, such as setting up servers, installing common programs and, of course, install a LAMP server.

 

 

Here is literally the only line you need to type in order to set up a LAMP server:

 

 

sudo tasksel install lamp-server

 

 

Why did I give up on setting up the LAMP server manually? I’ve decided that the level of headaches from setting up the server manually in Ubuntu was just not worth the amount of time that I was losing from not advancing in learning. Every minute I spent investigating why things weren’t working was making me jaded and contributed to the slowing pace and interest in completing the overall goal of writing my own blog in PHP.

 

I’ve always had an ongoing inner monologue about actually knowing everything about a certain task and knowing enough to be proficent at doing the task. For example, when I first started playing bass guitar in high school, I taught myself how to play specific songs using tabs which taught you where to put your fingers rather than learning musical theory. I got good enough at some songs to the point where I was able to play in front of my high school for a lunch time concert. However, I was overwhelmed with the constant feeling that I was a huge fraud and that I was basically guitar-heroing and not really a valid musician (a massive concern when you’re a kid in high school).

 

I ended up saying “fuck it” and sold my bass guitar on craigslist.

 

Flash foward to a couple years ago when I first started cycling. I was convinced that I wanted to build my own bike from scratch, until I learned that the holy grail of being a bicycle mechanic was the ability to build a wheel. Now, building a wheel is something that requires an insane amount of concentration and precision – but most of all experience.  Almost all tutorials of wheel building will stay that a wheel is properly tensioned when it “feels” right or, I shit you not, it “should be a G [musical note] with plain-gauge spokes and an A [musical note] with butted spokes” when plucked with your finger. Clearly, that is fucking insane.  After a couple of months, I once again was like “fuck it”, but instead of giving it up I just bought a set of wheels on ebay and was able to get on the road. Ever since then, I’ve become an avid cyclist.

 

I learned that you shouldn’t let the egotistical act of being an expert stand in the way of actually doing something. Hopefully, by building a LAMP stack the easy way (with 1 command) will allow me to actually continue to learn more about developing and keep moving foward.

Diving in face first with LAMP – Part 2: I only know how to do this command and the command immediately after

Ahh...to be a linux user.
Immediately, I’m running into issues with Linux.

 

I really like the command line interface, it really gives you a direct sense of what the computer is doing.  For example, if you’re using Microsoft Excel and trying to open a massive file, the app will just freeze and you’re not sure if the program is working or crashing. For me, the best part is that if I type in a command and I get the next line returned, it worked.  And it’s also really cool to type all this stuff and have your screen explode in text like you’re hacking the Pentagon. 

 

The downside for me is that if it doesn’t work, you’re shit out of luck.  Since I’m working off a tutorial, any outdated information or any sort of situation that isn’t outlined is going to completely derail this entire process.  But I guess that’s technically the fun part.

Continue reading →

Diving in face first with LAMP – Part 1: I literally just learned what this is

Brick Tamland loves LAMP

I had a conversation last week with a professional programmer named Ryan, who’s the husband of my friend Betsy.  As we were all talking about “the internet” over ice cream sandwiches and drinks, I had no choice but to ask the question that I think affects Gen-Yers today:

 

I have a lot of business ideas and thoughts for tech startups, but I can’t program.  Is it worth going back to school to learn Computer Science, knowing that even after I graduate I will not be as good as you are now, let alone at where your skill level will be in 2 to 4 years?

 

Continue reading →