Analytics@ – By For and From Data Science Professionals

Jan 23, 20: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

Warning: file_get_contents(http://events.analytics.club/tw/eventpull.php?cat=WEB): failed to open stream: HTTP request failed! in /home3/vishaltao/public_html/mytao/script/includeit.php on line 15

Warning: file_get_contents(http://news.analyticsweek.com/tw/newspull.php): failed to open stream: HTTP request failed! in /home3/vishaltao/public_html/mytao/script/includeit.php on line 15

[ COVER OF THE WEEK ]

Convincing Source

[ TIPS & TRICKS OF THE WEEK]

Data Have Meaning
We live in a Big Data world in which everything is quantified. While the emphasis of Big Data has been focused on distinguishing the three characteristics of data (the infamous three Vs), we need to be cognizant of the fact that data have meaning. That is, the numbers in your data represent something of interest, an outcome that is important to your business. The meaning of those numbers is about the veracity of your data.

[ DATA SCIENCE Q&A]

Q:Provide examples of machine-to-machine communications?
A: Telemedicine
– Heart patients wear specialized monitor which gather information regarding heart state
– The collected data is sent to an electronic implanted device which sends back electric shocks to the patient for correcting incorrect rhythms

Product restocking
– Vending machines are capable of messaging the distributor whenever an item is running out of stock

Source

[ VIDEO OF THE WEEK]

#FutureOfData Podcast: Conversation With Sean Naismith, Enova Decisions

Subscribe to Youtube

[ QUOTE OF THE WEEK]

Big Data is not the new oil. – Jer Thorp

[ PODCAST OF THE WEEK]

Harsh Tiwari talks about fabric of data driven leader in Financial Sector #FutureOfData #Podcast

iTunes GooglePlay

[ FACT OF THE WEEK]

We are seeing a massive growth in video and photo data, where every minute up to 300 hours of video are uploaded to YouTube alone.

Sourced from: Analytics.CLUB #WEB Newsletter

Build an Affordable $600 eSports Gaming PC to Play CS: GO, DotA 2, LoL and Overwatch

eSports games have become more and more popular among both fans and gamers. As a result, many people start dreaming of building eSports careersâmany gamers have pretty solid plans to achieve that goal. What does every gamer need to play eSports disciplines on the appropriate level, and to be able to challenge top LoL (League […]

The post Build an Affordable $600 eSports Gaming PC to Play CS: GO, DotA 2, LoL and Overwatch appeared first on TechSpective.

Source: Build an Affordable $600 eSports Gaming PC to Play CS: GO, DotA 2, LoL and Overwatch by administrator

Jan 16, 20: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

Warning: file_get_contents(http://news.analyticsweek.com/tw/newspull.php): failed to open stream: HTTP request failed! in /home3/vishaltao/public_html/mytao/script/includeit.php on line 15

[ COVER OF THE WEEK ]

Data shortage Source

[ AnalyticsWeek BYTES]

>> Visualizing taxi trips between NYC neighborhoods with Spark and Microsoft R Server by analyticsweek

>> The Wonders of Effectual Metadata Management: Automation by jelaniharper

>> Unraveling the Mystery of Big Data by v1shal

Wanna write? Click Here

[ FEATURED COURSE]

Artificial Intelligence

This course includes interactive demonstrations which are intended to stimulate interest and to help students gain intuition about how artificial intelligence methods work under a variety of circumstances…. more

[ FEATURED READ]

Hypothesis Testing: A Visual Introduction To Statistical Significance

Statistical significance is a way of determining if an outcome occurred by random chance, or did something cause that outcome to be different than the expected baseline. Statistical significance calculations find their … more

[ TIPS & TRICKS OF THE WEEK]

Data aids, not replace judgement
Data is a tool and means to help build a consensus to facilitate human decision-making but not replace it. Analysis converts data into information, information via context leads to insight. Insights lead to decision making which ultimately leads to outcomes that brings value. So, data is just the start, context and intuition plays a role.

[ DATA SCIENCE Q&A]

Q:Explain selection bias (with regard to a dataset, not variable selection). Why is it important? How can data management procedures such as missing data handling make it worse?
A: * Selection of individuals, groups or data for analysis in such a way that proper randomization is not achieved
Types:
– Sampling bias: systematic error due to a non-random sample of a population causing some members to be less likely to be included than others
– Time interval: a trial may terminated early at an extreme value (ethical reasons), but the extreme value is likely to be reached by the variable with the largest variance, even if all the variables have similar means
– Data: cherry picking, when specific subsets of the data are chosen to support a conclusion (citing examples of plane crashes as evidence of airline flight being unsafe, while the far more common example of flights that complete safely)
– Studies: performing experiments and reporting only the most favorable results
– Can lead to unaccurate or even erroneous conclusions
– Statistical methods can generally not overcome it

Why data handling make it worse?
– Example: individuals who know or suspect that they are HIV positive are less likely to participate in HIV surveys
– Missing data handling will increase this effect as its based on most HIV negative
-Prevalence estimates will be unaccurate

Source

[ VIDEO OF THE WEEK]

@CRGutowski from @GE_Digital on Using #Analytics to #Transform Sales #FutureOfData #Podcast

Subscribe to Youtube

[ QUOTE OF THE WEEK]

Hiding within those mounds of data is knowledge that could change the life of a patient, or change the world. Atul Butte, Stanford

[ PODCAST OF THE WEEK]

#BigData @AnalyticsWeek #FutureOfData #Podcast with @ScottZoldi, @FICO

iTunes GooglePlay

[ FACT OF THE WEEK]

According to estimates, the volume of business data worldwide, across all companies, doubles every 1.2 years.

Sourced from: Analytics.CLUB #WEB Newsletter

Where Chief Data Scientist & Open Source Meets – @dandegrazia #FutureOfData #Podcast

[youtube https://www.youtube.com/watch?v=-DkLqKwEhHo]

In this podcast @DanDeGrazia from @IBM spoke with @Vishaltx from @AnalyticsWeek to discuss the mingling of chief data scientist with open sources. He sheds light into some of the big opportunities in open source and how businesses could work together to achieve progress in data science. Dan also shared the importance of smooth communication for success as a data scientist.

Podcast Link:
iTunes: http://math.im/itunes
GooglePlay: http://math.im/gplay

Dan’s BIO:
Dan has almost 30 years working with large data sets. Starting with the unusual work of analyzing potential jury pools in the 1980s, Dan also did some of the first PC based voter registration analytics in the Chicago area including putting the first complete list of registered voters on a PC (as hard as that is to imagine today a 50 megabyte harddrive on DOS systems was staggering). Interested in almost anything new and technical, he worked at The Chicago Board of Trade where he taught himself BASIC to write algorithms while working as an Arbitrager in financial futures. After the military Dan moved to San Francisco where he worked several small companies and startups designing and implementing some of the first PC based fax systems (who cares now!), enterprise accounting software and working with early middleware connections using the early 3GL/4GL languages. Always perusing the technical edge cases Dan worked for InfoBright a Column store Database startup in the US and AMEA , at Lingotek an In-Q-Tel funded company working in large data set translations and big data analytics companies like Datameer and his current position as a Chief Data Scientist for Open Source in the IBM Channels organization. Dan’s current just for fun Project is working to create an app that will record and analyze bird songs and provide the user with information on the bird and the specifics of the current song.

About #Podcast:
#FutureOfData podcast is a conversation starter to bring leaders, influencers and lead practitioners to come on show and discuss their journey in creating the data driven future.

Want to sponsor?
Email us @ info@analyticsweek.com

Keywords:
#FutureOfData #DataAnalytics #Leadership #Podcast #BigData #Strategy

Source by v1shal

November 14, 2016 Health and Biotech analytics news roundup

Here’s the latest in health and biotech analytics:

Data Specifics Identified for Prediagnostic Heart Failure Detection: IBM researchers analyzed machine learning models that predict heart failure (paper). Among other findings, they worked out that models perform best with shorter prediction windows.

Will Google Take Over the Medical Industry? Big Questions at COâs Healthcare Conference: In the keynote speech at the Pulse Healthcare Conference, Andrew Quirk pointed to many new players entering the healthcare industry. Panels at the conference covered topics like patient experiences and the future of hospitals.

Accelerating cancer research with deep learning: Georgia Tourassi is head of Health Data Science at Oak Ridge National Laboratory. Her group is using deep neural networks to extract useful diagnostic data, such as the location of a tumor, from clinical reports.

A student innovation to tackle cognitive challenges in health informatics wins this yearâs Sysmex Award: The New Zealand diagnostics company gave the award to Daniel Surkalim, a University of Auckland student. He proposed using âgraphical relational integrated databasesâ to make it easier for providers to access electronic health data.

Originally Posted at: November 14, 2016 Health and Biotech analytics news roundup

Jan 09, 20: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

Warning: file_get_contents(http://news.analyticsweek.com/tw/newspull.php): failed to open stream: HTTP request failed! in /home3/vishaltao/public_html/mytao/script/includeit.php on line 15

[ COVER OF THE WEEK ]

Accuracy check Source

[ AnalyticsWeek BYTES]

>> AI Now 2019 report slams government use of facial recognition, biased AI by administrator

>> Talent Analytics: Old Wine In New Bottles? by analyticsweekpick

>> 12 Drivers of BigData Analytics by v1shal

Wanna write? Click Here

[ FEATURED COURSE]

Tackle Real Data Challenges

Learn scalable data management, evaluate big data technologies, and design effective visualizations…. more

[ FEATURED READ]

Hypothesis Testing: A Visual Introduction To Statistical Significance

[ TIPS & TRICKS OF THE WEEK]

[ DATA SCIENCE Q&A]

Q:You are compiling a report for user content uploaded every month and notice a spike in uploads in October. In particular, a spike in picture uploads. What might you think is the cause of this, and how would you test it?
A: * Halloween pictures?
* Look at uploads in countries that dont observe Halloween as a sort of counter-factual analysis
* Compare uploads mean in October and uploads means with September: hypothesis testing

Source

[ VIDEO OF THE WEEK]

Decision-Making: The Last Mile of Analytics and Visualization

Subscribe to Youtube

[ QUOTE OF THE WEEK]

The most valuable commodity I know of is information. Gordon Gekko

[ PODCAST OF THE WEEK]

Future of HR is more Relationship than Data – Scott Kramer @ValpoU #JobsOfFuture #Podcast

iTunes GooglePlay

[ FACT OF THE WEEK]

Distributed computing (performing computing tasks using a network of computers in the cloud) is very real. Google GOOGL -0.53% uses it every day to involve about 1,000 computers in answering a single search query, which takes no more than 0.2 seconds to complete.

Sourced from: Analytics.CLUB #WEB Newsletter

Making Magic with Treasure Data and Pandas for Python

Originally published on Treasure Data blog.

Magic functions, a mainstay of pandas, enable common tasks by saving you typing. Magic functions are functions preceeded by a % symbol. Magic functions have been introduced into pandas-td version 0.8.0! Toru Takahashi from Treasure Data walks us through.

Treasure Data’s magic functions work by wrapping a separate % around the original function, making the functions callable by %%. Letâs explore further to see how this works.

Until now

We start by creating a connection, importing our relevant libraries, and issuing a basic query, all from python (in Jupyter). Using the sample data, it would look like this:

import os
import pandas_td as td

#Initialize connection
con = td.connect(apikey=os.environ[âTD_API_KEYâ], endpoint = âhttps://api.treasuredata.comâ)
engine = con.query_engine(database=âsample_datasetsâ, type=âprestoâ)
#Read Treasure Data query into a DataFrame
df = td.read_td(âselect * from www_access, engineâ)

With the magic function

We can now do merely this:

%%td_use_sample_datasets

%%td_presto
select count(1) as cnt
from nasdaq

If you add the table name nasdaq after %% td_use, you can also see the schema:

Even better, you can tab edit the stored column names:

As long as %matplotlib inline is enabled; then you can throw a query into magicâs %%td_presto – -plot and immediately visualize it!

Very convenient!

How to enable it

Set the API_KEY environment variable:
export TD_API_KEY=1234/abcdâ¦

You can then load the magic comment automatically! Youâll want to save the following to ~/.ipython/profile_default/ipython_config.py

c = get_config()
c.InteractiveShellApp.extensions=[
âpandas_td.ipythonâ,
]

Let’s review

Loading your data:

Querying your data with presto:

Accessing stored columns:

Plotting:

Stay tuned for many more useful functions from pandas-td! These tools, including Pandas itself, as well as Python and Jupyter are always changing, so please let us know if anything is working differently than whatâs shown here.

Originally Posted at: Making Magic with Treasure Data and Pandas for Python by john-hammink

July 31, 2017 Health and Biotech analytics news roundup

Scientists use new data mining strategy to spot those at high Alzheimer’s risk: The researchers were able to split patients into different subgroups, which may help future clinical trials.

Amazon has a secret health tech team called 1492 working on medical records, virtual doc visits: The group is apparently looking both at new methods and leveraging current technology.

Protein Libraries Pave the Way for New Treatment Options: Researchers can make large numbers of proteins from the DNA that codes for them, enabling quicker study of molecular biological processes.

Collaborate or Collapse: Why Working Together is Essential for the Life Science Industry:Â There are substantial barriers to working together, but there are currently some initiatives to make it easier.

Originally Posted at: July 31, 2017 Health and Biotech analytics news roundup

Jan 02, 20: #AnalyticsClub #Newsletter (Events, Tips, News & more..)

Warning: file_get_contents(http://news.analyticsweek.com/tw/newspull.php): failed to open stream: HTTP request failed! in /home3/vishaltao/public_html/mytao/script/includeit.php on line 15

[ COVER OF THE WEEK ]

Complex data Source

[ FEATURED COURSE]

Introduction to Apache Spark

Learn the fundamentals and architecture of Apache Spark, the leading cluster-computing framework among professionals…. more

[ FEATURED READ]

Data Science from Scratch: First Principles with Python

Data science libraries, frameworks, modules, and toolkits are great for doing data science, but they’re also a good way to dive into the discipline without actually understanding data science. In this book, you’ll learn … more

[ TIPS & TRICKS OF THE WEEK]

Save yourself from zombie apocalypse from unscalable models
One living and breathing zombie in today’s analytical models is the pulsating absence of error bars. Not every model is scalable or holds ground with increasing data. Error bars that is tagged to almost every models should be duly calibrated. As business models rake in more data the error bars keep it sensible and in check. If error bars are not accounted for, we will make our models susceptible to failure leading us to halloween that we never wants to see.

[ DATA SCIENCE Q&A]

Q:Explain what a false positive and a false negative are. Why is it important these from each other? Provide examples when false positives are more important than false negatives, false negatives are more important than false positives and when these two types of errors are equally important
A: * False positive
Improperly reporting the presence of a condition when its not in reality. Example: HIV positive test when the patient is actually HIV negative

* False negative
Improperly reporting the absence of a condition when in reality its the case. Example: not detecting a disease when the patient has this disease.

When false positives are more important than false negatives:
– In a non-contagious disease, where treatment delay doesnt have any long-term consequences but the treatment itself is grueling
– HIV test: psychological impact

When false negatives are more important than false positives:
– If early treatment is important for good outcomes
– In quality control: a defective item passes through the cracks!
– Software testing: a test to catch a virus has failed

Source

[ VIDEO OF THE WEEK]

@Schmarzo @DellEMC on Ingredients of healthy #DataScience practice #FutureOfData #Podcast

Subscribe to Youtube

[ QUOTE OF THE WEEK]

Getting information off the Internet is like taking a drink from a firehose. Mitchell Kapor

[ PODCAST OF THE WEEK]

@AmyGershkoff on building #winning #DataScience #team #FutureOfData #Podcast

iTunes GooglePlay

[ FACT OF THE WEEK]

40% projected growth in global data generated per year vs. 5% growth in global IT spending.

Sourced from: Analytics.CLUB #WEB Newsletter

The Relationship Between Survey Response Rates and Survey Ratings

When soliciting feedback from customers through formal surveys, we only receive a percentage of completed or returned surveys. This percentage (number of people who answered the survey divided by the number of people in the sample) is referred to as the response or completion rate. In practice, I have seen response rates as low as 10% and as high as 80% across a variety of different surveys and target populations (e.g., employee and customer). How important is the response rate?

I recently got my hands on freeÂ US government data on patient survey ratings for over 3800 US hospitals. The Federal government, specifically theÂ Centers for Medicare & Medicaid ServicesÂ (CMS) and theÂ Agency for Healthcare Research and QualityÂ (AHRQ) funded the development of this standardized patient surveyÂ -Â HCAHPSÂ (Hospital Consumer Assessment of Healthcare Providers and Systems)Â – to publicly report the patientâs perspective of hospital care.

The HCAHPS data include a variety of data for each of the 3800 hospitals, including:

Patient ratings: The reported data reflect patient ratings of their inpatient experience across 10 different areas, eight touch points (e.g., nurse communication, pain management) and two loyalty-related questions (e.g., overall quality rating and recommend). Â Scores on these metrics can range from 0 (low) to 100 (high) and reflect the percent of patients who provided “top box” ratings. For the current analysis, I created a Patient Advocacy Loyalty index by averaging the two loyalty-related questions. I also used the other eight customer experience ratings.
Survey response rate: These data are reported as the simple response rate. I created five segments of hospitals based on their response rates. These five segments are: 1) 20% or less, 2) between 21% and 30%, 3) between 31% and 40%, 4) between 41% and 50% and 5) 51% or greater.
Number of completed surveys: This variable is reported as one of three levels: 1) less than 100 completed surveys, 2) 100-299 completed surveys and 3) 300 or more completed surveys.

Results

The average survey response rate across all 3848 hospitals was .32. That is, for every 100 patients who are asked to complete the survey, 32 actually provide feedback.

I compared patient advocacy ratings across the different levels of response rates and number of completed surveys. These analyses are visually depicted in Figure 1. As you can see, there are a couple of interesting findings:

Number of completed surveys is slightly related (RÂ² < .01)Â to patient loyalty. Hospitals that had less than 100 completed surveys had slightly higher patient loyalty scores than hospitals who had more than 100 completed surveys.
Response rate was strongly related (RÂ² = .32)Â to patient loyalty. Hospitals that had lower survey response rates hadÂ significantly and substantiallyÂ lower patient advocacy ratings compared to hospitals with higher survey response rates. In fact, there is about aÂ 25-point difference between hospitals with the lowest response rates (Patient Advocacy Loyalty ~ 60) and the highest response rates (Patient Advocacy Loyalty ~ 85). By the way, I found a similar pattern of results using the other patient experience metrics (see Figure 2); hospitals with lower response rates had patients who had poorer patient experiences compared to hospitals with higher response rates.

Patient Experience by Response Rates — Figure 2. Patient Experience Ratings by Response Rates

Why is there a relationship between survey response rate and survey ratings?Â PRC, a consulting firm that specializes in healthcare survey research,Â make the claim thatÂ response rates mayÂ causeÂ rating differences. They hint that, to improve your patient ratings, you need to have a higher response rate. While the representativeness of the sample of survey respondents to the population is paramount to drawing conclusions about the population, I am skeptical that merely improving your response rate will increase your ratings.

Perhaps response rate is just another measure of the quality of the customer/patient relationship.Â The findings suggest that patients who are dissatisfied with their hospital experience are less likely to complete a survey. If true, hospitals with truly dissatisfied patients will have lower ratings and lower response rates.

Potential Problems with HCAHPS Data?

The HCAHPS data are collected by many different survey vendors (In fact, there areÂ 44 approved survey vendorsÂ responsible for collecting the patient survey data) using three different data collection methods: 1) telephone only, 2) mail only and 3) mix mode (telephone and mail).Â There is some research that shows that methodological factors impact response rates. For example,Â two researchersÂ found a higher patient survey response rate forÂ face-to-faceÂ methods for recruitment (76.7%) or data collection (76.9%) compared to theÂ mailÂ method of recruitment (66.5%) or data collection (67%).

Using the HCAHPS patient ratings for hospital reimbursement purposes would require thatÂ differences across the various vendors/methods be minimal. It would be interesting (necessary?) to see if there are differences across 44 approved survey vendors and data collection methods with respect to the response rates, other survey process metrics and survey ratings. Understanding the reason behind the strong relationship between response rates and survey ratings is paramount to establishing the validity of the survey ratings.

Summary

Survey response rate was significantly and substantially related to survey ratings. Specifically, hospitals that had a higher survey response rate received higher patient ratings on their hospital experience. I will try to explore this issue in upcoming blog posts.

Large survey vendors may be in a good position to study the relationship between survey process measures (e.g., response rates) and survey ratings; these vendors have multiple accounts on which they have both types of metrics. It would be interesting to see if the current finding generalizes to other industries. Additionally, identifying the reasons behind the relationship between response rates and survey ratings would be essential to understanding the validity of the survey ratings.

Originally Posted at: The Relationship Between Survey Response Rates and Survey Ratings by bobehayes