Feda /dev/ - No Javascript allowed

https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html

INTRO
As I was browsing the web and catching up on some sites I visit periodically, I found a cool article from Tom Hayden about using Amazon Elastic Map Reduce (EMR) and mrjob in order to compute some statistics on win/loss ratios for chess games he downloaded from the millionbase archive, and generally have fun with EMR. Since the data volume was only about 1.75GB containing around 2 million chess games, I was skeptical of using Hadoop for the task, but I can understand his goal of learning and having fun with mrjob and EMR. Since the problem is basically just to look at the result lines of each file and aggregate the different results, it seems ideally suited to stream processing with shell commands.

I tried this out, and for the same amount of data I was able to use my laptop to get the results in about 12 seconds (processing speed of about 270MB/sec), while the Hadoop processing took about 26 minutes (processing speed of about 1.14MB/sec).

CONCLUSION
This find | xargs mawk | mawk pipeline gets us down to a runtime of about 12 seconds, or about 270MB/sec, which is around 235 times faster than the Hadoop implementation.

#20105 El guionista es el mismo, no esperemos mucho que luego las decepciones son más grandes.

ni con tus crocs

Buen cringe pero si alguien no sabe lo que es TCO debería irse a casa a meditar un poco, encontrar a Dios o alguna cosa similar de esas de moda.

Cuánta rabia dentro xd. A mí me parece algo cutre pero al menos le pusieron ganas.

Luego la mitad de aquí tenéis figuritas de naruto en el escritorio o cuadros de super mario.

La calvicie es el destino que nos aguarda a todos, cuanto antes asumais que el pelo es efímero antes alcanzaréis la tranquilidad. Also, calvo pero cabeza afeitada, que no hay que perder la dignidad.

#20107 Dios que puta vergüenza ajena xDDD.

#20121

#20123 cada dia de mi vida:

.

#20129 mis 250 pesta;as abiertas te saludan