GFY, real-life anecdote

That was the first phrase I learned from first people I met in San Francisco and first 24 hours in the US. After a nice day of exploring the city I came back to my hotel at fisherman's wharf. There were awesome sofas in front of fire pits outside. After coding for a while I was just sitting with a bottle of milk enjoying watching the fire.

Then a couple of ladies joined me. One of them told me about her husband, who died in a car accident, driving a motorcycle. My desire to buy a motorcycle slightly decreased... Then she told me to add her on facebook and went away.

Then a drunk guy asked me if he could join watching the fire, offered me a beer and made fun of me drinking milk (actually I didn't like it either, but there was no kefir around). We began talking about random things. After he learned that I am from Russia the conversation became so exciting that an old man joined our talk as well. The main question from them was about my attitude to USSR.

Actually I was born in USSR, but in several months since that time it collapsed.

Then they told me that their country is prospering because of ubiquitous competition, that people compete with each other even in being friends. You want to show that you are a better friend than the guy you are a friend with.

Dudes told funny stories about speculating with weed in 1970s, gave me a piece of advice to avoid girls from New Jersey.

Our conversation ended up with:
- Douchebag!
- Go fuck yourself!

Which was previously explained as the best way to express high respect to each other.

Success is Not Defined

I was writing a JS script interacting with an API. Last one had "success" as a response attribute (with true/false values). I forgot that it's a key, so I called "success" as a variable. When I ran the script it returned nothing, but an error: "success is not defined". That was the smartest statement I have ever recieved from a terminal. As for me it's almost true, I have no idea how to define success, it's something so temporary.

Books I've Read This Year

Recommended! Programming Collective Intelligence Building Smart Web 2.0 Applications (Toby Segaran)

This book helps you understand core data analysis techniques through solving real-world problems. Code samples could be better, but are pretty clear. Must read for those who want to implement smart features in their web applications.

Everyday Rails Testing with RSpec (Aaron Sumner)

Nice examples of how to use all DSL magick of RSpec. This book helps you build industrial-strength applications with Rails.

Express Web Application Development (Hage Yaapa)

The first part of this book is awesome. You will perfectly understand how to develop your Node.js application. But to be frank I expected more insights about ORMs and testing.

ggplot2: Elegant Graphics for Data Analysis (Hadley Wickham)

It doesn't matter wheather you use R or not. You should definetely know that ggplot2 exists and it can help you build beautiful charts with ease. Look up for ggplot2 examples and you'll understand what I am talking about.

High-Frequency Trading: A Practical Guide to Algorithmic Strategies and Trading Systems, 2nd Edition (Irene Aldridge)

Too much general information about how the market works and non-practical models further. However, you will understand what types of algorithmic trading exist and what their pros/cons are.

Recommended! Practical Object-Oriented Design in Ruby: An Agile Primer (Sandi Metz)

If you want to be a good programmer - read this book. You will (finally) understand how classes interact with each other, what is a factory pattern and how to make your code more readable.

Think Python: How to Think Like a Computer Scientist (Allen B. Downey)

In fact it's not about thinking like a computer scientist. You will animate turtles and learn basic python syntax. After all a good read for those who want to learn programming.

Redis in Action (Josiah L. Carlson)

This book is a collection of recipes, how to use Redis. I liked learning more about Redis persistence and administrating the database, but data structures description is poor - could be much better with more illustrations and challenges.

Algorithms Unlocked (Thomas H. Cormen)

This book is the one you can read in a bus. If you are a manager working with technical people - this book is a good choice to understand the basics. The book describes what algorithms are in general and where they are applied, no math inside.

Design Patterns in Ruby (Russ Olsen)

You should read this book after coding gets painful and boring. If you don't have enough experience it will be difficult for you to understand why all these paterns are valuable and should be used.

Eloquent Ruby (Russ Olsen)

One of the best sources of Ruby idioms I've met for now, but the problem is the same as it was in case of the book above.

Recommended! Brave New World (Aldous Huxley)

Everything written there is so true.

From Earth to the Moon (Jules Verne)

I like Jules Verne. His books empower my imagination. My dream is too read all his books, but it's difficult when you are in a hurry.

White Nights (Fyodor Dostoyevsky)

Too much interpersonal. I am not the fan of Dostoevsky, sorry.

Database Marketing: Analyzing and Managing Customers (Robert C. Blattberg)

There is machine learning, there are digital marketing metrics, there is software engineering. It's funny when you read a book, where authors try to cover all these things, but don't understand how they work in real life.

Gandhi An Autobiography: The Story of My Experiments With Truth (Mahatma Gandhi)

I read this book up to the moment, when Gandhi got 25. There are so many things you should be afraid to skip. I would definetely recommend you to read it.

Hackers & Painters: Big Ideas from the Computer Age (Paul Graham)

Cool ideas about the nature nerds and the purpose of schools.

Recommended! On Life (Leo Tolstoy)

Leo Tolstoy is one of my favourite philosophers. This book is full of wisdom and truth. By the way, Tolstoy was a friend with Ghandi.

How I learned D3

Everything genious is simple and D3.js is proving this. There is no other efficient way to build an unconventional plot except using this tool. It has a steep learning curve, but you gain a lot after you understand the approach.

The best way to learn D3 is reading all tutorials several times and then trying to replicate as many examples as possible. You can try my tutorial as well, it doesn't cover everything but is a good material for sure.

And here is the checklist you should keep in mind, before you start doing something really serious:

• margin conventions
• data enter/exit
• js callback functions
• transitions
• d3 helpers

Check out the latest speech by Mike Bostock, the father of D3. 


Eyeo 2013 - Mike Bostock from Eyeo Festival on Vimeo.

That's it. Good luck and have fun!

Redis for Data Analysis

Attractor has a lot of realtime calculations. In this post I would like to share my insights about working with streaming data using redis. There are many challenging issues related with realtime data analysis. Among them are the following:

• How to store data?
• Which data structures should be used?
• How to keep data persistent?
• How to make queries fast? 

The list is not an ideal one. However, it shows what you should keep in mind while building your own data warehouse with realtime data processing. 

You should definitely look through these presentations:

Kicking ass with redis from Dvir Volk

High-Volume Data Collection and Real Time Analytics Using Redis from cacois


My personal insights:

You should not use Redis like HDFS or other "big data" storage. Raw data can be stored in Redis, but not for a long time. Transfer raw data to other data storage solutions regularly.

Use all data structures efficiently. You should clearly understand when it is appropriate to use sorted sets, hashes or bitmaps. If you do everything right, Redis is able to handle billions of rows within milliseconds.

Don't forget about data persistence. Use AOF each second and keep db dumps in a safe place.

Unofficial Google Trends API

I was inspired by Sal Uryasev, who wrote the first unofficial Google Trends API and made it public. That's my fork of his repo: https://github.com/mac-r/unofficial-google-trends-api. The problem is that his code doesn't work anymore. I didn't like the idea of creating a new session as well. My script (https://github.com/mac-r/trendo) allows to get daily data per specified month without any authorization.

Few Words on HighlandDB

Almost a month ago I had a chance to release a lightweight NoSQL Database called HighlandDB. If you haven't heard about this gem (http://rubygems.org/gems/highland) - just watch this one minute video here.