Ttykuu

This is a problem I find a lot!! Can I achieve this goal without consuming so much time?

My code below achieves what I want it to achieve. However, I believe it could be a lot more efficient and Pythonic.

PROBLEM:
I want to extract summary data from a larger dataset and I only know how to do so utilizing next For loops. For example, I have a large dataset containing golf data, and I would like to extract summary statistics for the individual golf holes.

This code creates a scoring distribution and mean score for each Season-Hole-Round-Score vs. Par combination (48 rows in total).

import numpy as np
import pandas as pd
import itertools

seasons = [2001,2001,2001,2001,2002,2002,2002,2002]
holes = [1,1,2,2,1,1,2,2]
rounds = [3,4,3,4,3,4,3,4]
scores = [1,-1,0,0,0,1,-1,1] # actual scores vs. par

df = pd.DataFrame('season' : seasons, 'hole': holes, 'round':rounds, 'score': scores)


all_seasons = set(seasons); all_holes = set(holes); all_scores = [-1,0,1]
all_rounds = ["R3","R4","Weekend"] #some averages combine rounds
round_iter = np.arange(0,4) #position of rounds list
round_ids = [[3],[4],[3,4]] # weekend incldues rounds 3 and 4

hold_list = [] #blank list

for season,round,hole in itertools.product(all_seasons,round_iter,all_holes):

 hold_data = df[((df['season'] == season) & (df['hole'] == hole))
 & (df['round'].isin(round_ids[round-1]))]

 mean_score = hold_data['score'].mean()
 vspar_distro = hold_data['score'].value_counts().to_dict()
 for score in all_scores:
 count_score = 0
 if score in vspar_distro:
 count_score = vspar_distro[score]
 hold_list.append([season,all_rounds[round-1]
 ,hole,mean_score,score,count_score])


historical_df = pd.DataFrame(hold_list,columns 
 = ['season','round','hole','mean_score','vspar_score','count'])

This produces the df that I desire (here are the first 5 rows), but applying this to a file with 100k+ records takes a long time and I believe there is a more efficient way. Thanks!

 season round hole mean_score vspar_score count
0 2001 Weekend 1 0.0 -1 1
1 2001 Weekend 1 0.0 0 0
2 2001 Weekend 1 0.0 1 1
3 2001 Weekend 2 0.0 -1 0
4 2001 Weekend 2 0.0 0 2