Improving Self-Service Business Intelligence and Data Science

The heterogeneous complexities of big data present the foremost challenge in delivering that data to the end users who need them most. Those complexities are characterized by:

  • Disparate data sources: The influx of big data multiplied the sheer amount of data sources almost exponentially, including those both external and internal ones. Moreover, the quantity of sources required today are made more complex by…
  • Multiple technologies powering those sources: For almost every instance in which SQL is still deployed, there is seemingly another application, use case, or data source which involves an assortment of alternative technologies. Moreover, accounting for the plethora of technologies in use today is frequently aggravated by contemporary…
  • Architecture and infrastructure complications: With numerous advantages for deployments in the cloud, on-premise, and in hybrid manifestations of the two, contemporary enterprise architecture and infrastructure is increasingly ensnared in a process which protracts time to value for accessing data. The dilatory nature of this reality is only worsened in the wake of…
  • Heightened expectations for data: As data becomes ever entrenched in the personal lives of business users, the traditional lengthy periods of business intelligence and data insight are becoming less tolerable. According to Dremio Chief Marketing Officer Kelly Stirman, “In our personal lives, when we want to use data to answer questions, it’s just a few seconds away on Google…And then you get to work, and your experience is nothing like that. If you want to answer a question or want some piece of data, it’s a multi-week or multi-month process, and you have to ask IT for things. It’s frustrating as well.”

However, a number of recent developments have taken place within the ever-shifting data landscape to substantially accelerate self-service BI and certain aspects of data science. The end result is that despite the variegated factors characterizing today’s big data environments, “for a user, all of my data looks like it’s in a single high performance relational database,” Stirman revealed. “That’s exactly what every analytical tool was designed for. But behind the scenes, your data’s spread across hundreds of different systems and dozens of different technologies.”

Avoiding ETL

Conventional BI platforms were routinely hampered by the ETL process, a prerequisite for both integrating and loading data into tools with schema at variance with that of source systems. The ETL process was significant for three reasons. It was the traditional way of transforming data for application consumption. It was typically the part of the analytics process which absorbed a significant amount of time—and skill—because it required the manual writing of code. Furthermore, it resulted in multiple copies of data which could be extremely costly to organizations. Stirman observed that, “Each time you need a different set of transformations you’re making a different copy of the data. A big financial services institution that we spoke to recently said that on average they have eight copies of every piece of data, and that consumes about 40 percent of their entire IT budget which is over a billion dollars.” ETL is one of the facets of the data engineering process which monopolizes the time and resources of data scientists, who are frequently tasked with transforming data prior to leveraging them.

Modern self-service BI platforms eschew ETL with automated mechanisms that provide virtual (instead of physical) copies of data for transformation. Thus, each subsequent transformation is applied to the virtual replication of the data with swift in-memory technologies that not only accelerate the process, but eliminate the need to dedicate resources to physical copies. “We use a distributed process that can run on thousands of servers and take advantage of the aggregate RAM across thousands of servers,” Stirman said. “We can execute these transformations dynamically and give you a great high-performance experience on the data, even though we’re transforming it on the fly.” End users can enact this process visually without involving script.
Reflections

Today’s self-service BI and data science platforms have also expedited time to insight by making data more available than traditional solutions did. Virtual replications of datasets are useful in this regard because they are stored in the underlying BI solution—instead of in the actual source of data. Thus, these platforms can access that data without retrieving them from the initial data source and incurring the intrinsic delays associated with architectural complexities or slow source systems. According to Stirman, the more of these “copies of the data in a highly optimized format” such a self-service BI or data science solution has, the faster it is at retrieving relevant data for a query. Stirman noted this approach is similar to one used by Google, in which there are not only copies of web pages available but also “all these different ways of structuring data about the data, so when you ask a question they can give you an answer very quickly.” Self-service analytics solutions which optimize their data copies in this manner produce the same effect.

Prioritizing SQL

Competitive platforms in this space are able to account for the multiplicity of technologies the enterprise has to contend with in a holistic fashion. Furthermore, they’re able to do so by continuing to prioritize SQL as the preferred query language which is rewritten into the language relevant to the source data’s technology—even when it isn’t SQL. By rewriting SQL into the query language of the host of non-relational technology options, users effectively have “a single, unified future-proof way to query any data source,” Stirman said. Thus, they can effectively query any data source without understanding its technology or its query language, because the self-service BI platform does. In those instances in which “those sources have something you can’t express in SQL, we augment those capabilities with our distributed execution engine,” Stirman remarked.
User Experience

The crux of self-service platforms for BI and data science is that by eschewing ETL for quicker versions of transformation, leveraging in-memory technologies to access virtual copies of data, and re-writing queries from non-relational technologies into familiar relational ones, users can rely on their tool of choice for analytics. Business end users can choose from any popular Tableau, Qlik, or any other preferred tool, while data scientists can use R, Python, or any other popular data science platform. The fact that these solutions are able to facilitate these advantages at scale and in cloud environments adds to their viability. Consequently, “You log in as a consumer of data and you can see the data, and you can shape it the way you want to yourself without being able to program, without knowing these low level IT skills, and you get the data the way you want it through a powerful self-service model instead of asking IT to do it for you,” Stirman said. “That’s a fundamentally very different approach from the traditional approach.”

 

Source by jelaniharper

Nov 15, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

[  COVER OF THE WEEK ]

image
Statistics  Source

[ LOCAL EVENTS & SESSIONS]

More WEB events? Click Here

[ NEWS BYTES]

>>
 Why Big Data and Machine Learning are Essential for Cyber Security – insideBIGDATA Under  Big Data Analytics

>>
 Amazon’s Big Step Into IoT – Seeking Alpha Under  IOT

>>
 Hitachi and Tencent team up on ‘internet of things’ – Nikkei Asian Review Under  Internet Of Things

More NEWS ? Click Here

[ FEATURED COURSE]

Learning from data: Machine learning course

image

This is an introductory course in machine learning (ML) that covers the basic theory, algorithms, and applications. ML is a key technology in Big Data, and in many financial, medical, commercial, and scientific applicati… more

[ FEATURED READ]

Rise of the Robots: Technology and the Threat of a Jobless Future

image

What are the jobs of the future? How many will there be? And who will have them? As technology continues to accelerate and machines begin taking care of themselves, fewer people will be necessary. Artificial intelligence… more

[ TIPS & TRICKS OF THE WEEK]

Fix the Culture, spread awareness to get awareness
Adoption of analytics tools and capabilities has not yet caught up to industry standards. Talent has always been the bottleneck towards achieving the comparative enterprise adoption. One of the primal reason is lack of understanding and knowledge within the stakeholders. To facilitate wider adoption, data analytics leaders, users, and community members needs to step up to create awareness within the organization. An aware organization goes a long way in helping get quick buy-ins and better funding which ultimately leads to faster adoption. So be the voice that you want to hear from leadership.

[ DATA SCIENCE Q&A]

Q:Explain what a long-tailed distribution is and provide three examples of relevant phenomena that have long tails. Why are they important in classification and regression problems?
A: * In long tailed distributions, a high frequency population is followed by a low frequency population, which gradually tails off asymptotically
* Rule of thumb: majority of occurrences (more than half, and when Pareto principles applies, 80%) are accounted for by the first 20% items in the distribution
* The least frequently occurring 80% of items are more important as a proportion of the total population
* Zipf’s law, Pareto distribution, power laws

Examples:
1) Natural language
– Given some corpus of natural language – The frequency of any word is inversely proportional to its rank in the frequency table
– The most frequent word will occur twice as often as the second most frequent, three times as often as the third most frequent…
– The” accounts for 7% of all word occurrences (70000 over 1 million)
– ‘of” accounts for 3.5%, followed by ‘and”…
– Only 135 vocabulary items are needed to account for half the English corpus!

2. Allocation of wealth among individuals: the larger portion of the wealth of any society is controlled by a smaller percentage of the people

3. File size distribution of Internet Traffic

Additional: Hard disk error rates, values of oil reserves in a field (a few large fields, many small ones), sizes of sand particles, sizes of meteorites

Importance in classification and regression problems:
– Skewed distribution
– Which metrics to use? Accuracy paradox (classification), F-score, AUC
– Issue when using models that make assumptions on the linearity (linear regression): need to apply a monotone transformation on the data (logarithm, square root, sigmoid function…)
– Issue when sampling: your data becomes even more unbalanced! Using of stratified sampling of random sampling, SMOTE (‘Synthetic Minority Over-sampling Technique”, NV Chawla) or anomaly detection approach

Source

[ VIDEO OF THE WEEK]

@JustinBorgman on Running a data science startup, one decision at a time #Futureofdata #Podcast

 @JustinBorgman on Running a data science startup, one decision at a time #Futureofdata #Podcast

Subscribe to  Youtube

[ QUOTE OF THE WEEK]

Everybody gets so much information all day long that they lose their common sense. – Gertrude Stein

[ PODCAST OF THE WEEK]

@JohnTLangton from @Wolters_Kluwer discussed his #AI Lead Startup Journey #FutureOfData #Podcast

 @JohnTLangton from @Wolters_Kluwer discussed his #AI Lead Startup Journey #FutureOfData #Podcast

Subscribe 

iTunes  GooglePlay

[ FACT OF THE WEEK]

Brands and organizations on Facebook receive 34,722 Likes every minute of the day.

Sourced from: Analytics.CLUB #WEB Newsletter

To Trust A Bot or Not? Ethical Issues in AI

Given we see fake profiles and potentially chatbots that misfire and miscommunicate we would like your thoughts on whether there should be some sort of Government registry for robots so that consumers know they are legitimate or not. If we had a registry for trolls and or chatbots would that ensure that people could feel more comfortable that they are dealing with a legitimate business or would know if the profile or troll or bot is fake? Is it time for a good housekeeping seal of approval for AI?

These are all provocative questions and questions that are so new I am not sure there is one answer as they are so undefined. What do you think? Who should create such standards? Perhaps we should start by categorizing the types of AI?

Source by tony

Nov 08, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

[  COVER OF THE WEEK ]

image
Data security  Source

[ LOCAL EVENTS & SESSIONS]

More WEB events? Click Here

[ AnalyticsWeek BYTES]

>> The Modern Data Warehouse – Enterprise Data Curation for the Artificial Intelligence Future by analyticsweek

>> Analyzing Big Data: A Customer-Centric Approach by bobehayes

>> Analytics Implementation in 12 Steps: An Exhaustive Guide (Tracking Plan Included!) by analyticsweek

Wanna write? Click Here

[ NEWS BYTES]

>>
 Why Cloud (By Default) Gives You Security You Couldn’t Afford Otherwise – Forbes Under  Cloud Security

>>
  Under  Big Data Security

>>
 Make it so: Red River Mill Employees FCU rebrands as Engage – Credit Union Journal Under  Talent Analytics

More NEWS ? Click Here

[ FEATURED COURSE]

Baseball Data Wrangling with Vagrant, R, and Retrosheet

image

Analytics with the Chadwick tools, dplyr, and ggplot…. more

[ FEATURED READ]

The Signal and the Noise: Why So Many Predictions Fail–but Some Don’t

image

People love statistics. Statistics, however, do not always love them back. The Signal and the Noise, Nate Silver’s brilliant and elegant tour of the modern science-slash-art of forecasting, shows what happens when Big Da… more

[ TIPS & TRICKS OF THE WEEK]

Analytics Strategy that is Startup Compliant
With right tools, capturing data is easy but not being able to handle data could lead to chaos. One of the most reliable startup strategy for adopting data analytics is TUM or The Ultimate Metric. This is the metric that matters the most to your startup. Some advantages of TUM: It answers the most important business question, it cleans up your goals, it inspires innovation and helps you understand the entire quantified business.

[ DATA SCIENCE Q&A]

Q:Is it better to spend 5 days developing a 90% accurate solution, or 10 days for 100% accuracy? Depends on the context?
A: * “premature optimization is the root of all evils”
* At the beginning: quick-and-dirty model is better
* Optimization later
Other answer:
– Depends on the context
– Is error acceptable? Fraud detection, quality assurance

Source

[ VIDEO OF THE WEEK]

Surviving Internet of Things

 Surviving Internet of Things

Subscribe to  Youtube

[ QUOTE OF THE WEEK]

Data beats emotions. – Sean Rad, founder of Ad.ly

[ PODCAST OF THE WEEK]

#BigData #BigOpportunity in Big #HR by @MarcRind #JobsOfFuture #Podcast

 #BigData #BigOpportunity in Big #HR by @MarcRind #JobsOfFuture #Podcast

Subscribe 

iTunes  GooglePlay

[ FACT OF THE WEEK]

94% of Hadoop users perform analytics on large volumes of data not possible before; 88% analyze data in greater detail; while 82% can now retain more of their data.

Sourced from: Analytics.CLUB #WEB Newsletter

Three final talent tips: how to hire data scientists

This last post focuses on less tangible aspects, related to curiosity, clarity about what kind of data scientist you need, and having appropriate expectations when you hire.

8. Look for people with curiosity and a desire to solve problems

Radhika Kulkarni, PhD in Operations Research, Cornell University, teaching calculus as a grad student.

 

 

 

 

 

 

 

As I  blogged previously, Greta Roberts of Talent Analytics will tell you that the top traits to look for when hiring analytical talent are curiosity, creativity, and discipline, based on a study her organization did of data scientists. It is important to discover if your candidates have these traits, because they are necessary elements to find a practical solution and separate candidates from those who may get lost in theory. My boss Radhika Kulkarni, the VP of Advanced Analytics R&D at SAS, self-identified this pattern when she arrived at Cornell to pursue a PhD in math. This realization prompted her to switch to operations research, which she felt would allow her to pursue investigating practical solutions to problems, which she preferred to more theoretical research.

That passion continues today, as you can hear Radhika describe in this video on moving the world with advanced analytics. She says “We are not creating algorithms in an ivory tower and throwing it over the fence and expecting that somebody will use it someday. We actually want to build these methods, these new procedures and functionality to solve our customers’ problems.” This kind of practicality is another key trait to evaluate in your job candidates, in order to avoid the pitfall of hires who are obsessed with finding the “perfect” solution. Often, as Voltaire observed, “Perfect is the enemy of good.” Many leaders of analytical teams struggle with data scientists who haven’t yet learned this lesson. Beating a good model to death for that last bit of lift leads to diminishing returns, something few organizations can afford in an ever-more competitive environment. As an executive customer recently commented during the SAS Analytics Customer Advisory Board meeting, there is an “ongoing imperative to speed up that leads to a bias toward action over analysis. 80% is good enough.”

9. Think about what kind of data scientist you need

Ken Sanford, PhD in Economics, University of Kentucky, speaking about how economists make great data scientists at the 2014 National Association of Business Economists Annual Meeting. (Photo courtesy of NABE)

Ken Sanford describes himself as a talking geek, because he likes public speaking. And he’s good at it. But not all data scientists share his passion and talent for communication. This preference may or may not matter, depending on the requirements of the role. As this Harvard Business Review blog post points out, the output of some data scientists will be to other data scientists or to machines. If that is the case, you may not care if the data scientist you hire can speak well or explain technical concepts to business people. In a large organization or one with a deep specialization, you may just need a machine learning geek and not a talking one! But many organizations don’t have that luxury. They need their data scientists to be able to communicate their results to broader audiences. If this latter scenario sounds like your world, then look for someone with at least the interest and aptitude, if not yet fully developed, to explain technical concepts to non-technical audiences. Training and experience can work wonders to polish the skills of someone with the raw talent to communicate, but don’t assume that all your hires must have this skill.

10. Don’t expect your unicorns to grow their horns overnight

Annelies Tjetjep, M.Sc., Mathematical Statistics and Probability from the University of Sydney, eating frozen yogurt.

Annie Tjetjep relates development for data scientists to frozen yogurt, an analogy that illustrates how she shines as a quirky and creative thinker, in addition to working as an analytical consultant for SAS Australia. She regularly encounters customers looking for data scientists who have only chosen the title, without additional definition. She explains: “…potential employers who abide by the standard definitions of what a ‘data scientist’ is (basically equality on all dimensions) usually go into extended recruitment periods and almost always end up somewhat disappointed – whether immediately because they have to compromise on their vision or later on because they find the recruit to not be a good team player….We always talk in dimensions and checklists but has anyone thought of it as a cycle? Everyone enters the cycle at one dimension that they’re innately strongest or trained for and further develop skills of the other dimensions as they progress through the cycle – like frozen yoghurt swirling and building in a cup…. Maybe this story sounds familiar… An educated statistician who picks up the programming then creativity (which I call confidence), which improves modelling, then business that then improves modelling and creativity, then communication that then improves modelling, creativity, business and programming, but then chooses to focus on communication, business, programming and/or modelling – none of which can be done credibly in Analytics without having the other dimensions. The strengths in the dimensions were never equally strong at any given time except when they knew nothing or a bit of everything – neither option being very effective – who would want one layer of froyo? People evolve unequally and it takes time to develop all skills and even once you develop them you may choose not to actively retain all of them.”

So perhaps you hire someone with their first layer of froyo in place and expect them to add layers over time. In other words, don’t expect your data scientists to grow their unicorn horns overnight. You can build a great team if they have time to develop as Annie describes, but it is all about having appropriate expectations from the beginning.

To learn more, check out this series from SAS on data scientists, where you can read Patrick Hall’s post on the importance of keeping the science in data science, interviews with data scientists, and more.

And if you want to check out what a talking geek sounds like, Ken will be speaking at a National Association of Business Economists event next week in Boston – Big Data Analytics at Work: New Tools for Corporate and Industry Economics. He’ll share the stage with another talking geek, Patrick Hall, a SAS unicorn I wrote about it in my first post.

To read the original article on SAS, click here.

Originally Posted at: Three final talent tips: how to hire data scientists by analyticsweekpick

Free Research Report on the State of Patient Experience in US Hospitals

Download Free Report from TCELab: Improving the Patient Experience

The Centers for Medicare & Medicaid Services (CMS) will be using patient feedback about their care as part of their reimbursement plan for acute care hospitals (see Hospital Value-Based Purchasing (VBP) program). The purpose of the VBP program is to promote better clinical outcomes for patients and improve their experience of care during hospital stays. Not surprisingly, hospitals are focusing on improving the patient experience (PX) to ensure they receive the maximum of their incentive payments.

Free Download of Research Report on the Patient Experience

I spent the past few months conducting research on and writing about the importance of patient experience (PX) in US hospitals. My partners at TCELab have helped me summarize these studies into a single research report, Improving the Patient Experience . As far as I am aware, these series of studies are the first to integrate these disparate US hospital data sources (e.g., Patient Experience, Health Outcomes, Process of Care, and Medicare spending per patient) to apply predictive analytics for the purpose of identifying the reasons behind a loyal patient base.

While this research is really about the entirety of US hospitals, hospitals still need to dig deeper into their own specific patient experience data to understand what they need to do to improve the patient experience. This report is a good starting point for hospitals to learn what they need to do to improve the patient experience and increase patient loyalty. Read the entire press release about the research report, Improving the Patient Experience.

Get the free report from TCELab by clicking the image or link below:

Download Free Report from TCELab: Improving the Patient Experience

 

 

Originally Posted at: Free Research Report on the State of Patient Experience in US Hospitals by bobehayes

Nov 01, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

[  COVER OF THE WEEK ]

image
Trust the data  Source

[ LOCAL EVENTS & SESSIONS]

More WEB events? Click Here

[ AnalyticsWeek BYTES]

>> Can Police Use Data Science to Prevent Deadly Encounters? by analyticsweekpick

>> SDN and network function virtualization market worth $ 45.13 billion by 2020 by analyticsweekpick

>> Digital Marketing 2.0 – Rise of the predictive analytics by analyticsweekpick

Wanna write? Click Here

[ NEWS BYTES]

>>
 The Rise of Illiberal Artificial Intelligence – National Review Under  Artificial Intelligence

>>
 2018-2023 Prescriptive Analytics Market Overview, Growth, Types, Applications, Market Dynamics, Companies … – Stock Analysis Under  Prescriptive Analytics

>>
 KBI releases Kansas crime statistics for 2017 – WIBW News Now Under  Statistics

More NEWS ? Click Here

[ FEATURED COURSE]

CS229 – Machine Learning

image

This course provides a broad introduction to machine learning and statistical pattern recognition. … more

[ FEATURED READ]

Rise of the Robots: Technology and the Threat of a Jobless Future

image

What are the jobs of the future? How many will there be? And who will have them? As technology continues to accelerate and machines begin taking care of themselves, fewer people will be necessary. Artificial intelligence… more

[ TIPS & TRICKS OF THE WEEK]

Save yourself from zombie apocalypse from unscalable models
One living and breathing zombie in today’s analytical models is the pulsating absence of error bars. Not every model is scalable or holds ground with increasing data. Error bars that is tagged to almost every models should be duly calibrated. As business models rake in more data the error bars keep it sensible and in check. If error bars are not accounted for, we will make our models susceptible to failure leading us to halloween that we never wants to see.

[ DATA SCIENCE Q&A]

Q:Provide examples of machine-to-machine communications?
A: Telemedicine
– Heart patients wear specialized monitor which gather information regarding heart state
– The collected data is sent to an electronic implanted device which sends back electric shocks to the patient for correcting incorrect rhythms

Product restocking
– Vending machines are capable of messaging the distributor whenever an item is running out of stock

Source

[ VIDEO OF THE WEEK]

@AnalyticsWeek: Big Data at Work: Paul Sonderegger

 @AnalyticsWeek: Big Data at Work: Paul Sonderegger

Subscribe to  Youtube

[ QUOTE OF THE WEEK]

We chose it because we deal with huge amounts of data. Besides, it sounds really cool. – Larry Page

[ PODCAST OF THE WEEK]

@JohnTLangton from @Wolters_Kluwer discussed his #AI Lead Startup Journey #FutureOfData #Podcast

 @JohnTLangton from @Wolters_Kluwer discussed his #AI Lead Startup Journey #FutureOfData #Podcast

Subscribe 

iTunes  GooglePlay

[ FACT OF THE WEEK]

Decoding the human genome originally took 10 years to process; now it can be achieved in one week.

Sourced from: Analytics.CLUB #WEB Newsletter

Uber: When Big Data Threatens Local Democracy

3043801-poster-p-1-heres-exactly-when-normal-taxis-are-cheaper-than-uber-data-scientists-reveal-all

Big data is threatening to crush local democracy across the country–and if it succeeds, it may distort local transit and infrastructure development for decades to come.

As Uber has sought to dominate the local taxi industry from Delhi to New York City, the company has deployed its multi-billion dollar venture capital war chest to fight politicians across the country and world, often ignoring local laws as it introduced its app and drivers into the heavily regulated taxi industry.  In New York City, a bill has been introduced to limit the growth of the company locally while the City Council studies the implications for the local taxi industry.

Yesterday, Uber added an attack ad against the City’s mayor Bill De Blasio on the front page of its hailing app, melding its attempt to control local taxi service with seeking control of local politics.  In doing so, it highlights the danger of letting multi-billion dollar global corporations control any part of local transit or other infrastructure, since it gives them a stake in distorting local politics as well.

Uber may be a young company but they have entered old-style politics with a vengeance, hiring David Plouffe, the former strategist for President Obama’s 2008 election campaign to help direct a team of 250 lobbyists operating in at least 50 cities and states around the country.  On top of vast financial resources for traditional lobbying, they control an equally important resource – data and communication with voters throughout local constituencies.  In local political fights, Uber has used email and its app real estate to launch multiple attacks on political opponents.

An Opening Salvo in the Politics of Local Logistics

The fight over Uber is not about who runs local taxis, but really about who will control local transit and related infrastructure in the future.  Uber has made it clear its ambitions go far beyond taxis to encompass what Inc. magazine calls “the future of logistics.”

The data Uber collects on users and local transportation can be converted into delivery services or, as writer Ken Roose explains, “like Amazon, it can become something akin to an all-purpose utility–it’ll just be a way you get things and go places.” Uber has already launched a prototype “Uber Cargo” delivery business in Hong Kong and food delivery and courier services in other cities.

This ties into plans by companies like Google and Tesla to introduce driverless cars and a rush of tech companies to control the logistics and information related to local economies and commerce.   Driverless taxis are the obvious long-term next step for a company like Uber and could reshape urban transportation as fundamentally as the original introduction of the automobile.

The big data companies are already gearing up for the politics of controlling the next generation of urban infrastructure.  Google for example now spends more on lobbying that any other company, a large part of it ($18.2 million) on federal lobbying, but the company has also built a network of state lobbyists to help it on local fights like legalizing its driverless car project.

Driverless cars combined with Uber data – and Google is a major investor in Uber – could remake urban transit, especially as local laws and infrastructure are changed to accommodate them.   Analysts are already discussing how the “‘transportation cloud’…will quickly become dominant form of transportation – displacing far more than just car ownership, it will take the majority of users away from public transportation as well.”

History of Global Corporations Distorting Local Transit Development

With so much at stake, the danger of letting big data money gut local democratic decision-making is obvious.  We only need look to the history of how the auto industry used its political muscle to literally pave the way for destroying local mass transit in multiple cities and pushing highways and suburbanization.  Some of that movement was going to happen naturally, but the auto companies sped the process along and deepened it with actions such as buying up local trolley systems and converting them to bus systems.

City streets, which had been a shared resource of cars, bikes and pedestrians, were converted to car-only use. Where car drivers were once held criminally liable for any pedestrian killed by their car, car companies launched major lobbying campaigns to create a new crime, “jaywalking” that put responsibility on pedestrians not to be in the cars’ way.

Like Uber, the car companies used the communication infrastructure of the day, in that case wire services for reporters, to seize control of the public debate on use of streets.  The National Automobile Chamber of Commerce encouraged reporters to send basic details of traffic accidents to their service and would receive back a complete article to print the next day, with the articles shifting the blame for accidents to pedestrians.

The result of decades of car industry lobbying was the gutting of much of urban America.

Fighting Monopoly as a Political Problem

With momentous political decisions facing local governments as big data, driverless cars and other technologies reshape local transit and logistics, we need to worry about big data monopolies not only distorting local industries at the economic level but also how their political power may distort political decision-making as well.

Uber is backed by an array of economic players, from Google to Goldman Sachs and local politicians should recognize that legalizing Uber is not just adding an additional economic player to the local economy.  It will add a political player willing to spend billions of dollars with the goal of establishing a global logistics behemoth–and seemingly willing to waste any politicians who get in its way.

If Uber is going to misuse its economic power to try to control local political institutions — including real estate on its apps as political attack ads — local governments should feel justified in restricting its growth until both the potential economic and political problems of an emerging taxi monopolist are addressed.  And it raises the broader issue of how big data’s political power needs to be restrained to ensure communities get to decide how best to use technology, rather than the technology companies deciding how best to use communities for their own economic interests.

Note: This article originally appeared in Huffington Post. Click for link here.

Originally Posted at: Uber: When Big Data Threatens Local Democracy by analyticsweekpick

Janet Amos Pribanic says: ‘Business Analytics – It’s really OK that it’s not perfect first time out!’

Janet Amos Pribanic
Chief Operating Officer
John Daniel Associates, Inc.
Janet’s Profile

Business Analytics is changing rapidly. Traditional BI is being challenged due to the rate at which we are not only collecting data, but wanting to leverage that data for business advantage.

A recent survey of over 40 technology vendors’ clients by one of the largest IT research firms showed that ultimately, the customer experience matters. The value given to the customer is what matters, always! So, how do we get there?….

Even if not perfect – get it in the hands of the business fast

When we evaluate successful analytics customers, many of them started with a very inexpensive (hear free trial) solution, and leveraged that experience to build a successful solution and architecture. What better way to learn, than to fail (or partially fail) and take that experience forward. The term “fail” here does not necessarily mean a solution has been built and thrown away. It means that what has been learned via a trial process or prototype has given us valuable insight on what is most meaningful moving forward with analytics and successful analytics architecture. A functional architecture, or methodology, is the best place to start. Here are the 5 steps:

Functional BI Architecture

  1. Identify the business problem we are out to solve with data (analytics)
  2. Gather the data
  3. Build the model to support the business
  4. View and explore the data
  5. Deploy the operational insights

And note – this is an iterative process. There is value in getting this out there not exactly right. However, get it out there fast – you will leverage feedback faster, and this will allow better insight from the business because they can see it and take corrective action faster. In addition, disparities are exposed faster, incorrect business rules are exposed faster, and by having that exposure, the organization can now take action and leverage powerful and cheap architectures, like Hadoop, to enable you to take in massive amounts of data and store data very inexpensively. Now you are prepared to take on strategic business analytics.

Let’s look at a possible example. You have identified and gathered data around a problem in your supply chain. After gathering the data, the next move is to explore it. You sample the data in the data set, test it and analyze it. You develop hypotheses which are then tested. For example, you might do analysis to figure out which two or three vendors are late in deliveries, resulting in customer satisfaction issues.

After you determine why those vendors are not performing, you operationalize the insights you’ve gained, building them into your business logic and workflows. If ABC Company is consistently late on deliveries within the supply chain and contributes to two actions that map to the supply chain model (challenge), begin corrective action before affecting additional clients.

Once insights are operationalized, it is important to close the model feedback loop. The models you build could (and likely will) change over time; the factors that caused supply chain challenge this year may not hold a year from now, as market and other factors change. Test your hypotheses to see if the model still holds or if it needs adjustment. For example, as new forms of vendor interaction are introduced, the supply chain variables may also change over time.

Advanced analytics should become the mainstay of your secret sauce. The point is to use all of your resources effectively: data modelers and the many people with business domain knowledge.

Get analytics out there fast, even if not perfect: it’s a counterintuitive way to make rapid progress. Take in only as much data as you need, analyze and create operational models, and refine those models. Then take those models and begin to build your functional architecture.

To deliver successful analytics, you will also need to plan and staff correctly. To do advanced analytics at scale, there are two approaches from a staffing perspective:

  1. Hire lots of expensive data modelers
  2. Leverage people who are technically savvy in your company with strong business acumen

The first way is certainly challenging. In fact, it can’t scale either because there is not an abundant supply of data modelers to hire, even if you had unlimited resources.

The second way takes a different and proven successful approach. Do not be constrained by the technology that enables analytics but instead focus on the strength of logic that powers analytics.

In practical terms, ask data modelers to partner with you to research ways to solve business problems, and then have them build consumable models that solve those problems. With models that serve as templates, businesspeople can perform analytics on their own, working with the models the data modelers developed, and then extend those models to new areas as business logic dictates.

Your company will significantly benefit by this secret sauce. Choose to make it part of your core competency!

The post Janet Amos Pribanic says: ‘Business Analytics – It’s really OK that it’s not perfect first time out!’ appeared first on John Daniel Associates, Inc..

Source by analyticsweek

Oct 25, 18: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

[  COVER OF THE WEEK ]

image
Trust the data  Source

[ LOCAL EVENTS & SESSIONS]

More WEB events? Click Here

[ AnalyticsWeek BYTES]

>> The convoluted world of data scientist by v1shal

>> Ten Guidelines for Clean Customer Feedback Data by bobehayes

>> Why Cloud-native is more than software just running on someone else’s computer by analyticsweekpick

Wanna write? Click Here

[ NEWS BYTES]

>>
 How the Windows 10 October 2018 update will impact your enterprise IoT deployments – TechRepublic Under  IOT

>>
 Global Advanced Analytics Market research report 2018: Techniques, Region, Feature Analysis, Study Methodology … – IDA Report Under  Social Analytics

>>
 Hybrid cloud data specialist Datrium nabs $60M led by Samsung at a $282M valuation – TechCrunch Under  Hybrid Cloud

More NEWS ? Click Here

[ FEATURED COURSE]

Machine Learning

image

6.867 is an introductory course on machine learning which gives an overview of many concepts, techniques, and algorithms in machine learning, beginning with topics such as classification and linear regression and ending … more

[ FEATURED READ]

The Future of the Professions: How Technology Will Transform the Work of Human Experts

image

This book predicts the decline of today’s professions and describes the people and systems that will replace them. In an Internet society, according to Richard Susskind and Daniel Susskind, we will neither need nor want … more

[ TIPS & TRICKS OF THE WEEK]

Keeping Biases Checked during the last mile of decision making
Today a data driven leader, a data scientist or a data driven expert is always put to test by helping his team solve a problem using his skills and expertise. Believe it or not but a part of that decision tree is derived from the intuition that adds a bias in our judgement that makes the suggestions tainted. Most skilled professionals do understand and handle the biases well, but in few cases, we give into tiny traps and could find ourselves trapped in those biases which impairs the judgement. So, it is important that we keep the intuition bias in check when working on a data problem.

[ DATA SCIENCE Q&A]

Q:What does NLP stand for?
A: * Interaction with human (natural) and computers languages
* Involves natural language understanding

Major tasks:
– Machine translation
– Question answering: “what’s the capital of Canada?”
– Sentiment analysis: extract subjective information from a set of documents, identify trends or public opinions in the social media

– Information retrieval

Source

[ VIDEO OF THE WEEK]

@Schmarzo @DellEMC on Ingredients of healthy #DataScience practice #FutureOfData #Podcast

 @Schmarzo @DellEMC on Ingredients of healthy #DataScience practice #FutureOfData #Podcast

Subscribe to  Youtube

[ QUOTE OF THE WEEK]

Torture the data, and it will confess to anything. – Ronald Coase

[ PODCAST OF THE WEEK]

@DrewConway on fabric of an IOT Startup #FutureOfData #Podcast

 @DrewConway on fabric of an IOT Startup #FutureOfData #Podcast

Subscribe 

iTunes  GooglePlay

[ FACT OF THE WEEK]

39 percent of marketers say that their data is collected ‘too infrequently or not real-time enough.’

Sourced from: Analytics.CLUB #WEB Newsletter