DISCONTINUED -- PROJECT NOT MAINTAINED

Uber Crawler / Usage Analytics

@version alpha-0.2.2 | Branch | Build Status | Coverage | | ------ | ------------ | -------- | | master | | | | alpha-0.2.3 | | |

Synopsis

Uber web interface crawler - Convert the trips table into a CSV file

Installation & Configuration

Minimum Requirements

- PHP (5.6+)
  - XDebug is a requirement for running the unit tests

Installation is very basic, just:

• Clone this repository into any directory:

git clone https://github.com/Link-/uber_data.git

• Install dependencies and build the autoload file:

composer install

• Build your App.php configuration file:

Using CLI

This repository ships with a handy command-line interface companion named uberc - located at ./bin/uberc

• Add ./bin to your path with

export PATH="$PATH:<project path>/bin"

• Configure (this has to be done only once)

uberc config

• Analyze: Will generate the analytics files in the desired directories specified at the config step

uberc analyze

Sample Output

2016-06-03,Logan,$7.73,uberX,Los Angeles,N.A
  2016-06-03,John,$14.45,uberX,Los Angeles,N.A
  2016-06-02,Mark,$4.70,uberX,Los Angeles,N.A
  2016-06-02,Logan,Canceled,uberX,Los Angeles,N.A
  2016-06-02,Morgan,$13.23,uberX,Los Angeles,N.A
  2016-06-01,Sleimann,$4.79,uberX,Los Angeles,N.A
  2016-06-01,George,$14.36,uberX,Los Angeles,N.A

Jupyter Notebook

Installation & Configuration

Minimum Requirements

python3 (3.4.3)
  pip3 (1.5.4)
  jupyter (4.1.0)
  pandas (0.18.1)
  matplotlib (1.5.1)

Review the installation requirements / steps per depedency by following the reference links provided below.

• Install python3, you will need a C compiler and the Python headers and finally pip3:

sudo apt-get install python3 build-essential python3-dev python3-setuptools python3-pip

• Verify that python3 and pip3 have been downloaded / installed:

pip3 -V
    pip 1.5.4 from /usr/lib/python3/dist-packages (python 3.4)
        
    python3 -V
    Python 3.4.3

• Install Jupyter

sudo pip3 install jupyter

• Install pandas -- usually numpy gets bundled with pandas but just in case, install it separately (link to the installation guide below)

sudo pip3 install pandas

• Install matplotlib

sudo apt-get install python3-matplotlib
    # Upgrade to v.1.5.1

Installation Guides

• pip : installation guide
• jupyter : installation guide
• pandas : installation guide
• scipy (numpy) : installation guide
• matplotlib : installation guide

Execution

• Run jupyter notebook:

jupyter notebook

• Open the Uber-DataAnalysis-0.1.ipynb found in uberdata/analysis/

• In the 3rd row, change the value of file_location as per the below:

# FROM
    filelocation = r'<path to uberdata>/sampledata/sample_data.csv'
    
    # TO
    file_location = r'<path to uber data>/data/<the file created by the crawler>.csv'

• Press Cell then Run All from the menubar

• Voila, you should game the output as shown in the Sample Analysis Output

Sample Analysis Output

Uber Data Anlysis v0.1 Notebook: Uber-DataAnalysis-0.1.ipynb