Skip to main content

Comments

Post a Comment

Popular posts from this blog

Installing pyspark with Jupyter

Installing pyspark with Jupyter Check List Python is a wonderful programming language for data analytics. Normally, I prefer to write python codes inside   Jupyter Notebook  (previous known as  IPython ), because it allows us to create and share documents that contain live code, equations, visualizations and explanatory text.  Apache Spark  is a fast and general engine for large-scale data processing.  PySpark  is the Python API for Spark. So it’s a good start point to write PySpark codes inside jupyter if you are interested in data science: IPYTHON_OPTS="notebook" pyspark --master spark://localhost:7077 --executor-memory 7g Install Jupyter If you are a pythoner, I highly recommend installing  Anaconda . Anaconda conveniently installs Python, the Jupyter Notebook, and other commonly used packages for scientific computing and data science. Go to  https://www.continuum.io/downloads , find the ins...
1 comment
Read more

Spark A to Z

Spark A to Z Simplicity, Flexibility and Performance are the major advantages of using Spark . Criteria Hadoop MapReduce Apache Spark Memory  Does not leverage the memory of the hadoop cluster to maximum. Let's save data on memory with the use of RDD's. Disk usage MapReduce is disk oriented. Spark caches data in-memory and ensures low latency. Processing Only batch processing is supported Supports real-time processing through spark streaming. Installation Is bound to hadoop. Is not bound to Hadoop. ·  Spark is 100 times faster than Hadoop for big data processing as it stores the data in-memory, by placing it in Resilient Distributed Databases (RDD). ·  Spark is easier to program as it comes with an interactive mode. ·  It provides complete recovery using lineage graph whenever something goes wrong. high availability in Apache Spark ·  Implementing single node recovery with local file system ·  Using Sta...
1 comment
Read more
10.9% of Android devices run OS 4.0 Ice Cream Sandwich By  Ian Hardy  on July 3, 2012 at 8:01am in  Mobile News Google introduced Android OS 4.0 Ice Cream Sandwich on October 18th. Now, almost 9 months later, the OS has officially reached double diget numbers when it comes to the percentage of Android devices running a particular version. According to the Android Developers site ICS now powers 10.9% of device,  up from 7.1% a month ago . This increase is probably from all the new 4.0 powered devices, such as the HTC One X, One S and One V, plus the Samsung Galaxy S III. Unfortunately the OS that still has the lions share users is the 15-month old 2.3 Gingerbread with 64%, dropping 1% over last month. Cupcake (v1.5) and Donut (v1.6) continue to drop off the face of the world and now represent a combined percentage of 0.7%. Next month should be better for Ice Cream Sandwich numbers, plus we’ll probably see first stats on OS 4.1 Jelly Bean – which will be comin...
Post a Comment
Read more