Building a data strategy in 2016 – Vicki Boykis | CapTech | @vboykis
Building a data strategy in 2016 – Vicki Boykis | CapTech | @vboykis
0 0
senior-dev-day-talk
Senior Dev Day Talk SlidesOn Github veekaybee / senior-dev-day-talk
Building a data strategy in 2016
Vicki Boykis | CapTech | @vboykis
About CapTech
About me
The data landscape in 2006
The data landscape today
Now that we have all this cool stuff, how do we use it?
Agenda
- The basic rules of data
- Picking the right tool
- Common architecture patterns today
Basic rules of data
Data rule #1: Law of entropy
Natural processes have a preferred progress towards chaos
Goal: "I want to know how many customers I have."
Data Rule #1a: Vicki's Rule of Data
The further away from source your data is, the more annoying it gets to query and manage
Result
Data Rule #2: Pareto Rule
People use 20% of their tools 80% of the time because people stick to what they know.From the Data Science Salary Survey 2015
and The emergence of Spark.Data Rule #3: Conway's Law
Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations
How to evaluate data tools
Q1: Do you have enough data for Big Data?
If you don't have big data, here's what your stack looks like:
Q2: What kind of data do you have?
- Slow/fast : stock trading v.s. home insurance
- Relational/ non-relational: banking transactions v.s. Wikipedia
- Is your data special? : Are you stargazing?
Q3: Who is going to be working with your data?
- Engineers - Documentation? APIs? Testing?
- Analysts - Access? Lineage?
- Decision-makers - Data meaning?
Common architecture today
New data trends
Infrastructure Analysis Visualization Kafka Tensor Flow Caravel Feather Bayesian Query Language Jupyter DashboardsThanks! Questions?
For more, check out my data post.
About the Icons:
Twitter: @vboykis /Web
Building a data strategy in 2016Vicki Boykis | CapTech | @vboykis