B W0d<@sdZddlmZddlmZddlZddlmZmZddl Z ddl m Z ddl Z ddl mZmZmZmZmZmZmZmZmZmZmZddlZddlZddlmZdd lmZm Z ddl!m"m#Z$dd l%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-dd l.m/Z0dd l1m2Z2dd l3m4Z4m5Z5m6Z6m7Z7ddl8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@ddlAmBZBmCZCddlDmEZEddlFmGmHZHddlImJZJmKZKmLZLmMZMddlNmOZOmPZPmQZQddlRmGmSZTddlUmVZVddlWmXZXddlYmZZZm[Z[m\Z\ddl]m^Z^m_Z_m`Z`ddlambZbddlcmdZdddlemfZfddlgmhZhmiZier4ddl mjZjdZkdddd Zld!Zmd"Znd#Zod$Zpe-Gd%d&d&ePZqed'd(d)d*d+ZreeeeeegefeeegefeeeffZsGd,d-d-ePeQe(Zted.eXd/ZuGd0d'd'ete(Zve7evd;d3d4d5d6d7d7d7d7d7d7d7d'd8 d9d:ZwdS)>> df = pd.DataFrame({'A': 'a a b'.split(), ... 'B': [1,2,3], ... 'C': [4,6,5]}) >>> g = df.groupby('A') Notice that ``g`` has two groups, ``a`` and ``b``. Calling `apply` in various ways, we can get different grouping results: Example 1: below the function passed to `apply` takes a DataFrame as its argument and returns a DataFrame. `apply` combines the result for each group together into a new DataFrame: >>> g[['B', 'C']].apply(lambda x: x / x.sum()) B C 0 0.333333 0.4 1 0.666667 0.6 2 1.000000 1.0 Example 2: The function passed to `apply` takes a DataFrame as its argument and returns a Series. `apply` combines the result for each group together into a new DataFrame. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``. >>> g[['B', 'C']].apply(lambda x: x.astype(float).max() - x.min()) B C A a 1.0 2.0 b 0.0 0.0 Example 3: The function passed to `apply` takes a DataFrame as its argument and returns a scalar. `apply` combines the result for each group together into a Series, including setting the index as appropriate: >>> g.apply(lambda x: x.C.max() - x.B.min()) A a 5 b 2 dtype: int64a >>> s = pd.Series([0, 1, 2], index='a a b'.split()) >>> g = s.groupby(s.index) From ``s`` above we can see that ``g`` has two groups, ``a`` and ``b``. Calling `apply` in various ways, we can get different grouping results: Example 1: The function passed to `apply` takes a Series as its argument and returns a Series. `apply` combines the result for each group together into a new Series. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``. >>> g.apply(lambda x: x*2 if x.name == 'a' else x/2) a 0.0 a 2.0 b 1.0 dtype: float64 Example 2: The function passed to `apply` takes a Series as its argument and returns a scalar. `apply` combines the result for each group together into a Series, including setting the index as appropriate: >>> g.apply(lambda x: x.max() - x.min()) a 1 b 0 dtype: int64)templatedataframe_examplesZseries_examplesa Compute {fname} of group values. Parameters ---------- numeric_only : bool, default {no} Include only float, int, boolean columns. If None, will attempt to use everything, then use only numeric data. min_count : int, default {mc} The required number of valid values to perform the operation. If fewer than ``min_count`` non-NA values are present the result will be NA. Returns ------- Series or DataFrame Computed {fname} of values within each group. aa Apply a function `func` with arguments to this %(klass)s object and return the function's result. Use `.pipe` when you want to improve readability by chaining together functions that expect Series, DataFrames, GroupBy or Resampler objects. Instead of writing >>> h(g(f(df.groupby('group')), arg1=a), arg2=b, arg3=c) # doctest: +SKIP You can write >>> (df.groupby('group') ... .pipe(f) ... .pipe(g, arg1=a) ... .pipe(h, arg2=b, arg3=c)) # doctest: +SKIP which is much more readable. Parameters ---------- func : callable or tuple of (callable, str) Function to apply to this %(klass)s object or, alternatively, a `(callable, data_keyword)` tuple where `data_keyword` is a string indicating the keyword of `callable` that expects the %(klass)s object. args : iterable, optional Positional arguments passed into `func`. kwargs : dict, optional A dictionary of keyword arguments passed into `func`. Returns ------- object : the return type of `func`. See Also -------- Series.pipe : Apply a function with arguments to a series. DataFrame.pipe: Apply a function with arguments to a dataframe. apply : Apply function to each group instead of to the full %(klass)s object. Notes ----- See more `here `_ Examples -------- %(examples)s a Call function producing a like-indexed %(klass)s on each group and return a %(klass)s having the same indexes as the original object filled with the transformed values Parameters ---------- f : function Function to apply to each group. Can also accept a Numba JIT function with ``engine='numba'`` specified. If the ``'numba'`` engine is chosen, the function must be a user defined function with ``values`` and ``index`` as the first and second arguments respectively in the function signature. Each group's index will be passed to the user defined function and optionally available for use. .. versionchanged:: 1.1.0 *args Positional arguments to pass to func. engine : str, default None * ``'cython'`` : Runs the function through C-extensions from cython. * ``'numba'`` : Runs the function through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or the global setting ``compute.use_numba`` .. versionadded:: 1.1.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{'nopython': True, 'nogil': False, 'parallel': False}`` and will be applied to the function .. versionadded:: 1.1.0 **kwargs Keyword arguments to be passed into func. Returns ------- %(klass)s See Also -------- %(klass)s.groupby.apply : Apply function ``func`` group-wise and combine the results together. %(klass)s.groupby.aggregate : Aggregate using one or more operations over the specified axis. %(klass)s.transform : Call ``func`` on self producing a %(klass)s with transformed values. Notes ----- Each group is endowed the attribute 'name' in case you need to know which group you are working on. The current implementation imposes three requirements on f: * f must return a value that either has the same shape as the input subframe or can be broadcast to the shape of the input subframe. For example, if `f` returns a scalar it will be broadcast to have the same shape as the input subframe. * if this is a DataFrame, f must support application column-by-column in the subframe. If f also supports application to the entire subframe, then a fast path is used starting from the second chunk. * f must not mutate groups. Mutation is not supported and may produce unexpected results. See :ref:`gotchas.udf-mutation` for more details. When using ``engine='numba'``, there will be no "fall back" behavior internally. The group data and group index will be passed as numpy arrays to the JITed user defined function, and no alternative execution attempts will be tried. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, see the examples below. Examples -------- >>> df = pd.DataFrame({'A' : ['foo', 'bar', 'foo', 'bar', ... 'foo', 'bar'], ... 'B' : ['one', 'one', 'two', 'three', ... 'two', 'two'], ... 'C' : [1, 5, 5, 2, 5, 5], ... 'D' : [2.0, 5., 8., 1., 2., 9.]}) >>> grouped = df.groupby('A') >>> grouped.transform(lambda x: (x - x.mean()) / x.std()) C D 0 -1.154701 -0.577350 1 0.577350 0.000000 2 0.577350 1.154701 3 -1.154701 -1.000000 4 0.577350 -0.577350 5 0.577350 1.000000 Broadcast result of the transformation >>> grouped.transform(lambda x: x.max() - x.min()) C D 0 4 6.0 1 3 8.0 2 4 6.0 3 3 8.0 4 4 6.0 5 3 8.0 .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, for example: >>> grouped[['C', 'D']].transform(lambda x: x.astype(int).max()) C D 0 5 8 1 5 9 2 5 8 3 5 9 4 5 8 5 5 9 a Aggregate using one or more operations over the specified axis. Parameters ---------- func : function, str, list or dict Function to use for aggregating the data. If a function, must either work when passed a {klass} or when passed to {klass}.apply. Accepted combinations are: - function - string function name - list of functions and/or function names, e.g. ``[np.sum, 'mean']`` - dict of axis labels -> functions, function names or list of such. Can also accept a Numba JIT function with ``engine='numba'`` specified. Only passing a single function is supported with this engine. If the ``'numba'`` engine is chosen, the function must be a user defined function with ``values`` and ``index`` as the first and second arguments respectively in the function signature. Each group's index will be passed to the user defined function and optionally available for use. .. versionchanged:: 1.1.0 *args Positional arguments to pass to func. engine : str, default None * ``'cython'`` : Runs the function through C-extensions from cython. * ``'numba'`` : Runs the function through JIT compiled code from numba. * ``None`` : Defaults to ``'cython'`` or globally setting ``compute.use_numba`` .. versionadded:: 1.1.0 engine_kwargs : dict, default None * For ``'cython'`` engine, there are no accepted ``engine_kwargs`` * For ``'numba'`` engine, the engine can accept ``nopython``, ``nogil`` and ``parallel`` dictionary keys. The values must either be ``True`` or ``False``. The default ``engine_kwargs`` for the ``'numba'`` engine is ``{{'nopython': True, 'nogil': False, 'parallel': False}}`` and will be applied to the function .. versionadded:: 1.1.0 **kwargs Keyword arguments to be passed into func. Returns ------- {klass} See Also -------- {klass}.groupby.apply : Apply function func group-wise and combine the results together. {klass}.groupby.transform : Aggregate using one or more operations over the specified axis. {klass}.aggregate : Transforms the Series on each group based on the given function. Notes ----- When using ``engine='numba'``, there will be no "fall back" behavior internally. The group data and group index will be passed as numpy arrays to the JITed user defined function, and no alternative execution attempts will be tried. Functions that mutate the passed object can produce unexpected behavior or errors and are not supported. See :ref:`gotchas.udf-mutation` for more details. .. versionchanged:: 1.3.0 The resulting dtype will reflect the return value of the passed ``func``, see the examples below. {examples}c@s4eZdZdZddddZddZdd d d Zd S) GroupByPlotzE Class implementing the .plot attribute for groupby objects. GroupBy)groupbycCs ||_dS)N)_groupby)selfrGrJM/var/www/html/venv/lib/python3.7/site-packages/pandas/core/groupby/groupby.py__init__szGroupByPlot.__init__cs fdd}d|_|j|S)Ncs |jS)N)plot)rI)argskwargsrJrKfszGroupByPlot.__call__..frM)__name__rHapply)rIrNrOrPrJ)rNrOrK__call__szGroupByPlot.__call__str)namecsfdd}|S)Ncsfdd}j|S)Ncst|jS)N)getattrrM)rI)rNrOrUrJrKrPsz0GroupByPlot.__getattr__..attr..f)rHrR)rNrOrP)rUrI)rNrOrKattrsz%GroupByPlot.__getattr__..attrrJ)rIrUrWrJ)rUrIrK __getattr__szGroupByPlot.__getattr__N)rQ __module__ __qualname____doc__rLrSrXrJrJrJrKrE srErFzIterator[GroupBy])rGreturnccs"|z |VWd|XdS)z2 Set / reset the group_selection_context. N)_set_group_selection_reset_group_selection)rGrJrJrKgroup_selection_context&s r_c@sTeZdZUdZded<eZded<ejdddd d d d d dddddh BZded<ded <ded <e ddddZ e ddddZ e e ddddZ e e dddd Ze e d!d"Ze d#d$Ze d%d&Ze ed'd(Ze d)dd*d+Zed,ed-d.eed/d0d1d2d3Ze eZe d:d4dd5d6Ze d7dd8d9ZdS); BaseGroupByNzIndexLabel | None_group_selectionzfrozenset[str]_apply_allowlistas_indexaxisdropna exclusionsgrouper group_keyskeyslevelmutatedobjobservedsortsqueezeintzops.BaseGrouperbool)r\cCs t|jS)N)lengroups)rIrJrJrK__len__RszBaseGroupBy.__len__rTcCs t|S)N)object__repr__)rIrJrJrKrvVszBaseGroupBy.__repr__zdict[Hashable, np.ndarray]cCs|jjS)z4 Dict {group name -> group labels}. )rgrs)rIrJrJrKrs[szBaseGroupBy.groupscCs|jjS)N)rgngroups)rIrJrJrKrwcszBaseGroupBy.ngroupscCs|jjS)z5 Dict {group name -> group indices}. )rgindices)rIrJrJrKrxhszBaseGroupBy.indicesc sddt|dkrgStjdkr6ttj}nd}|d}t|trt|tsbd}t|t|t|ksyfdd|DStk r}zd}t||Wdd}~XYnXfd d|Dfd d |D}n|fd d |D}fd d|DS)zd Safe get multiple indices, translate keys for datelike to underlying repr. cSs4t|tjrddSt|tjr(ddSddSdS)NcSst|S)N)r)keyrJrJrK{zABaseGroupBy._get_indices..get_converter..cSs t|jS)N)rZasm8)ryrJrJrKrz}r{cSs|S)NrJ)ryrJrJrKrzr{) isinstancedatetimenpZ datetime64)srJrJrK get_converterws   z/BaseGroupBy._get_indices..get_converterrNzsz,BaseGroupBy._get_indices..zHmust supply a same-length tuple to get_group with multiple grouping keyscsg|] }|qSrJrJ)rr)rrJrKrsc3s&|]}tddt|DVqdS)css|]\}}||VqdS)NrJ)rrPnrJrJrK sz5BaseGroupBy._get_indices...N)tuplezip)rrU) convertersrJrKrsz+BaseGroupBy._get_indices..c3s|]}|VqdS)NrJ)rrU) converterrJrKrscsg|]}j|gqSrJ)rxget)rrU)rIrJrKrs)rrrxnextiterr|r ValueErrorKeyError)rInamesZ index_sampleZ name_samplemsgerrrJ)rrrrIrK _get_indicesps,   zBaseGroupBy._get_indicescCs||gdS)zQ Safe get index, translate keys for datelike to underlying repr. r)r)rIrUrJrJrK _get_indexszBaseGroupBy._get_indexcCsB|jdkst|jtr2|jdk r,|j|jS|jS|j|jSdS)N) _selectionr|rlr>ra)rIrJrJrK _selected_objs   zBaseGroupBy._selected_objzset[str]cCs|j|jBS)N)rl_dir_additionsrb)rIrJrJrKrszBaseGroupBy._dir_additionsrFa >>> df = pd.DataFrame({'A': 'a b a b'.split(), 'B': [1, 2, 3, 4]}) >>> df A B 0 a 1 1 b 2 2 a 3 3 b 4 To get the difference between each groups maximum and minimum value in one pass, you can do >>> df.groupby('A').pipe(lambda x: x.max() - x.min()) B A a 2 b 2)klassexamplesz/Callable[..., T] | tuple[Callable[..., T], str]r)funcr\cOstj||f||S)N)compipe)rIrrNrOrJrJrKrszBaseGroupBy.pipercCs8|dkr|j}||}t|s(t||j||jdS)a Construct DataFrame from group with provided name. Parameters ---------- name : object The name of the group to get as a DataFrame. obj : DataFrame, default None The DataFrame to take the DataFrame out of. If it is None, the object groupby was called on will be used. Returns ------- group : same type as obj N)rd)rrrrrZ_take_with_is_copyrd)rIrUrlZindsrJrJrK get_groups  zBaseGroupBy.get_groupz(Iterator[tuple[Hashable, FrameOrSeries]]cCs|jj|j|jdS)z Groupby iterator. Returns ------- Generator yielding sequence of (name, subsetted object) for each group )rd)rg get_iteratorrlrd)rIrJrJrK__iter__s zBaseGroupBy.__iter__)N)rQrYrZra__annotations__ frozensetrbr3Z _hidden_attrsrrtrvpropertyrsrwrxrrr!rrr rr_pipe_templaterrErMrrrJrJrJrKr`;sR   3   r`OutputFrameOrSeries)boundc@szeZdZUdZded<ded<edd d d d ddd ddddddddddZddddZedddddZeddddZ edddd Z d!dd"d#Z eddd$d%d&Z ed'd'd(d)d*Z d+d,d-d.Zd+d,d/d0Zddd$d1d2Zd3dd4d5d6Zed7d8Zedd9d:d;Zedd9djd?ed@dAdBdCZedDdEdEdFdGdHZedIdJZeddd dddLdMdNZdOd ddOdPdQdRZddddd dSdTdUZdddd dVdWdXZedddYdZd[Zed\d]Z eddd^d_d`daZ!ee"dbddcddZ#ededfZ$ee%dgdhee&dddidjdkZ'ee%dgdhee&dddidldmZ(e%dgdhee&dndoZ)ee%dgdhe%e&dpe*j+fd3dqdrdsZ,ee%dgdhee&e*j+fd3dqdtduZ-ee%dgdhee&dd dwdxdyZ.ee%dgdhee&dd dwdzd{Z/ee%dgdhee&dd dwd|d}Z0ee%dgdhee&dEdd~dZ1ee2e3dddde*j+dfd3d dddZ4ee2e3dddde*j+dfd3d dddZ5ee2e3dd dKdddd dddZ6ee2e3dd dKdddd dddZ7ee2e3dd dKdddd dddZ8ee2e3dd dKdddd dddZ9ee%dgdhee&ddddZ:ee2e;j<ddZee%dgdhee&ddZ?ee%dgdhee&ddZ@edddddZAee%dgdhdddZBeBZCee%dgdhdddZDeDZEee%dgdhe%e&dpdddddddZFedddddZGee%dgdhdddddZHee%dgdhdddddZIee%dgdhee&dddddd dddZJee%dgdhee&dddZKee%dgdhee&dddZLee%dgdhee&ddd„ZMee%dgdhee&dddĄZNed e*j+d d d d dd d d ddf dddd3dddddddddǜ ddɄZOee%dgdhddd˄ZPee%dgdhee&ddd΄ZQee%dgdhe%e&dpdddфZRee%dgdhe%e&dpdddӄZSeeTjUfd'dd'd՜ddׄZVeddddddڜdd܄ZWdS)rFa Class for grouping and aggregating relational data. See aggregate, transform, and apply functions on this object. It's easiest to use obj.groupby(...) to use GroupBy, but you can also do: :: grouped = groupby(obj, ...) Parameters ---------- obj : pandas object axis : int, default 0 level : int, default None Level of MultiIndex groupings : list of Grouping objects Most users should ignore this exclusions : array-like, optional List of columns to exclude name : str Most users should ignore this Returns ------- **Attributes** groups : dict {group name -> group labels} len(grouped) : int Number of groups Notes ----- After grouping, see aggregate, apply, and transform functions. Here are some other brief notes about usage. When grouping by multiple groups, the result index will be a MultiIndex (hierarchical) by default. Iteration produces (key, group) tuples, i.e. chunking the data by group. So you can write code like: :: grouped = obj.groupby(keys, axis=axis) for key, group in grouped: # do something with the data Function calls on GroupBy, if not specially implemented, "dispatch" to the grouped data. So if you group a DataFrame and wish to invoke the std() method on each group, you can simply do: :: df.groupby(mapper).std() rather than :: df.groupby(mapper).aggregate(np.std) You can pass arguments to these "wrapped" functions, too. See the online documentation for full exposition on these topics and much more zops.BaseGrouperrgrqrcNrTFrz_KeysArgType | NonerpzIndexLabel | Nonezops.BaseGrouper | Nonezfrozenset[Hashable] | None)rlrirdrjrgrf selectionrcrnrhrormrkrec Cs||_t|tstt|||_|sHt|ts8td|dkrHtd||_ ||_ | |_ | |_ | |_ | |_| |_||_|dkrddlm}|||||| | |j|jd\}}}||_|||_||_|rt|nt|_dS)Nz(as_index=False only valid with DataFramerz$as_index=False only valid for axis=0) get_grouper)rdrjrnrmrkre)rr|r6AssertionErrortyperjr5 TypeErrorrrcrirnrhrormrkrepandas.core.groupby.grouperrrlZ_get_axis_numberrdrgrrf)rIrlrirdrjrgrfrrcrnrhrormrkrerrJrJrKrLNs>   zGroupBy.__init__rT)rWcCsH||jkrt||S||jkr(||Stdt|jd|ddS)N'z' object has no attribute ')Z_internal_names_setru__getattribute__rlAttributeErrorrrQ)rIrWrJrJrKrXs    zGroupBy.__getattr__r)rUr\c sjkstt0tjttjsBfddSWdQRXtt jt fdd}|_ |S)Ncs t|S)N)rV)rI)rUrJrKrzr{z'GroupBy._make_wrapper..cs\djkr$dddkr$jd<fdd}|_tjkrN|S|jS)Nrdcs|fS)NrJ)x)rNrPrOrJrKcurriedsz7GroupBy._make_wrapper..wrapper..curried) parametersrrdrQr7Zplotting_methodsrR_python_apply_general_obj_with_exclusions)rNrOr)rPrUrIsig)rNrOrKwrappers    z&GroupBy._make_wrapper..wrapper) rbrr_rVrr|types MethodTyperRrinspect signaturerQ)rIrUrrJ)rPrUrIrrK _make_wrappers    zGroupBy._make_wrapperNone)r\cCsv|j}|jr,|jdk r,|jjdkr,|jdks0dSdd|jD}t|rr|jj}|jt |dd |_| ddS)z Create group based selection. Used when selection is not passed directly but instead via a grouper. NOTE: this should be paired with a call to _reset_group_selection NcSs"g|]}|jdkr|jr|jqS)N)rjin_axisrU)rgrJrJrKrsz0GroupBy._set_group_selection..F)rnr) rgrc groupingsrlndimrarrZ _info_axis differencer;tolist _reset_cache)rIgrpZgroupersaxrJrJrKr]s    zGroupBy._set_group_selectioncCs|jdk rd|_|ddS)z Clear group based selection. Used for methods needing to return info on each group regardless of whether a group selection was previously set. Nr)rar)rIrJrJrKr^s zGroupBy._reset_group_selectionzIterable[Series]cCs t|dS)N)r)rIrJrJrK_iterate_slicesszGroupBy._iterate_slices)not_indexed_samecsdddlm}fdd}|s||jd}jrFddjjn jj}|jr|jj |s|j |j \}} t |}|j|jd}n|j|jdd }nzjr||}jr|} jj} jj} ||j| | | dd }n ttt|}||j|d }n||}||jd}jjd kr>jjnj} t|tr`| dk r`| |_|S) Nr)concatcs,x&tj|D]}|j}|q W|S)N)rZnot_none _get_axisrdZ_reset_identity)valuesvr)rIrJrKreset_identitys  z/GroupBy._concat_objects..reset_identity)rdcSsdS)NTrJ)rrJrJrKrzr{z)GroupBy._concat_objects..F)rdcopy)rdrilevelsrrn)rdrir)pandas.core.reshape.concatrrdrefilteraxesrrZhas_duplicatesequalsindexZget_indexer_non_unique_values algorithmsZunique1dtakereindexrhrcrgrrlistrangerrrlrrUrr|r>)rIrirrrrresultrZindexer_rhZ group_levelsZ group_namesrUrJ)rIrK_concat_objectss@    zGroupBy._concat_objectsr)rr\cCs|jjr(|j|j|j|jdd|Stt| |jj }|j||jdd|j |jd}t |j t |jj k}|r|j }|jj ||_ n|j|j|j|jdd|S)NT)rdZinplace)rd)rgZ is_monotonicZset_axisrlrrdr;r~ concatenater result_index sort_indexrrrr)rIrZoriginal_positionsZ dropped_rowsZsorted_indexerrJrJrK_set_result_index_ordered-sz!GroupBy._set_result_index_orderedz"Mapping[base.OutputKey, ArrayLike])outputcCs t|dS)N)r)rIrrJrJrK_wrap_aggregated_outputLszGroupBy._wrap_aggregated_outputcCs t|dS)N)r)rIrrJrJrK_wrap_transformed_outputOsz GroupBy._wrap_transformed_outputcCs t|dS)N)r)rIdatarirrrJrJrK_wrap_applied_outputRszGroupBy._wrap_applied_outputzbool | lib.NoDefault) numeric_onlyr\cCs`|tjkr\|jjdkrXd}|jr*|jj}n|j}|}t|j r\t|j s\|j s\d}nd}|S)at Determine subclass-specific default value for 'numeric_only'. For SeriesGroupBy we want the default to be False (to match Series behavior). For DataFrameGroupBy we want it to be True (for backwards-compat). Parameters ---------- numeric_only : bool or lib.no_default Returns ------- bool TF) r no_defaultrlrrdrrZ_get_numeric_datarrcolumnsempty)rIrrlcheckrJrJrK_resolve_numeric_onlyUs   zGroupBy._resolve_numeric_onlyc Csxt|std|jj\}}}t||}tj||dd}|j||jd }|j | } t ||\} } | | | |fS)Nz5Numba engine can only be used with a single function.F)Z allow_fill)rd) callableNotImplementedErrorrg group_infor?rtake_ndrrdto_numpyrrZgenerate_slices) rIrridsrrw sorted_indexZ sorted_ids sorted_dataZsorted_index_datastartsendsrJrJrK _numba_preps zGroupBy._numba_prep) engine_kwargscOsz|||\}}}} |j} t|||} | | |||t| t|jf|} |df} | tkrf| t| <| jt |ddS)a( Perform groupby transform routine with the numba engine. This routine mimics the data splitting routine of the DataSplitter class to generate the indices of each group in the sorted data and then passes the data and indices into a Numba jitted function. Zgroupby_transformr)rd) rrg_get_group_keysr8Zgenerate_numba_transform_funcrrrr@rr~Zargsort)rIrrrrNrOrrrrrhZnumba_transform_funcr cache_keyrJrJrK_transform_with_numbas    zGroupBy._transform_with_numbacOs|||\}}}} |j} t|||} | | |||t| t|jf|} |df} | tkrf| t| <|jjdkrt j | |jj d}nt | |jj dd}| |fS)a* Perform groupby aggregation routine with the numba engine. This routine mimics the data splitting routine of the DataSplitter class to generate the indices of each group in the sorted data and then passes the data and indices into a Numba jitted function. Z groupby_aggr)rr)rU) rrgrr8Zgenerate_numba_agg_funcrrrr@Znkeysr< from_tuplesrr;)rIrrrrNrOrrrrrhZnumba_agg_funcrrrrJrJrK_aggregate_with_numbas$    zGroupBy._aggregate_with_numbarCZ dataframerD)inputrc stsr\tr4tfdd}q`ttdrRttd}q`tdn}tddLy| ||j }Wn2t k rt || ||j SQRXYnXWdQRX|S)Nc s(tjdd|fSQRXdS)Nignore)all)r~Zerrstate)r)rNrrOrJrKrPszGroupBy.apply..fnanz6func must be a callable if args or kwargs are suppliedzmode.chained_assignment) ris_builtin_funcrrhasattrr-rVrrrrrr_)rIrrNrOrPrrJ)rNrrOrKrRs    $z GroupBy.applyrr)rPrr\cCs0|j|||j\}}}|j||||p*|jdS)a- Apply function f in python space Parameters ---------- f : callable Function to apply data : Series or DataFrame Data to apply f to Returns ------- Series or DataFrame data after applying f )r)rgrRrdrrk)rIrPrrirrkrJrJrKr szGroupBy._python_apply_generalc stfdd}i}|jdkr6|||jSx|t|D]l\}}|j}y|j ||} Wn4t k rt j dt |jdtddwDYnXtj||d} | || <qDW|s|||jS||S) Ncs|fS)NrJ)r)rNrrOrJrKrz&r{z-GroupBy._python_agg_general..rzDropping invalid columns in z.agg is deprecated. In a future version, a TypeError will be raised. Before calling .agg, select only columns which should be valid for the aggregating function.) stacklevel)labelposition)rrrwrr enumeraterrUrg agg_seriesrwarningswarnrrQ FutureWarningr7 OutputKeyr) rIrrNrOrPridxrlrUrryrJ)rNrrOrK_python_agg_general#s(   zGroupBy._python_agg_general)r min_countaliasnpfuncc Cs6t|$|j||||d}|j|jddSQRXdS)N)howaltrrrG)method)r__cython_agg_general __finalize__rl)rIrrrrrrJrJrK _agg_generalHs zGroupBy._agg_generalr)rrrr\cCs~|jdkrt|}n.t|j}|jddks0t|jdddf}|jj||dd}t |t rrt |j ||j d}t||dS)zn Fallback to pure-python aggregation if _cython_operation raises NotImplementedError. rNrT)Zpreserve_dtype)dtype)r)rr>r5rshaperilocrgrr|r0rZ_from_sequencerr=)rIrrrZserdf res_valuesrJrJrK_agg_py_fallback\s    zGroupBy._agg_py_fallback)rrrrcCs t|dS)N)r)rIrrrrrJrJrKrszGroupBy._cython_agg_general)rrrdcKs t|dS)N)r)rIrrrdrOrJrJrK_cython_transformszGroupBy._cython_transform)enginerc Osdt|rt| |j}WdQRX|jdkr0|n|}|j||f|d|i|}|jjdkr|tt|jj ||j |j dStt |jj | |j |jdSt|p|}t|ts|j|f||S|tjkrd|d} t| nz|tjks|tjkrt||||St|ddt||||}WdQRX||rN||S|j|f||SdS) Nrr)rr)rrUrz2' is not a valid function name for transform(name)rmT)rAr_rrZto_framerrlrr5 _constructorrrr>ZravelrUrZget_cython_funcr|rTZ_transform_generalr7Ztransform_kernel_allowlistrZcythonized_kernelsZtransformation_kernelsrV temp_setattrZ_can_use_transform_fastZ_wrap_transform_fast_result) rIrrrrNrOrrrrrJrJrK _transforms0           zGroupBy._transformcCst|dkrtjgdd}ntt|}|rD|jj||jd}n^tjt|jj t d}| dd|| t <t|t|jjdddgj}|j|}|S)Nrint64)r)rdFTr)rrr~arrayrnrrrrdrrrqfillastyperpZtilerrrwhere)rIrxrefilteredmaskrJrJrK _apply_filters  $ zGroupBy._apply_filterz np.ndarray) ascendingr\c Cs|jj\}}}t||}||t|}}|dkrBtjdtjdStjd|dd|ddkf}ttjt |d|f}| } |r| t | ||8} n&t | tj|dddf|| } tj|tj d} tj |tj d| |<| | jtjddS) a. Parameters ---------- ascending : bool, default True If False, number in reverse, from length of group - 1 to 0. Notes ----- this is currently implementing sort=False (though the default is sort=True) for groupby in general r)rTNrrF)r)rgrr?rrr~rr#Zr_diffZnonzerocumsumrepeatintparanger&) rIr+rrrwZsortercountrunrepoutrevrJrJrK_cumcount_arrays  " &zGroupBy._cumcount_arrayz type[Series]cCs,t|jtr|jjSt|jts$t|jjS)N)r|rlr5Z_constructor_slicedr>rr )rIrJrJrK_obj_1d_constructors zGroupBy._obj_1d_constructorc sRdddfdd }dddd dd d d }|jd ddttjddd|||d S)zO Shared func to call any / all Cython GroupBy implementations. rztuple[np.ndarray, type])valsr\csnt|r8r"tdd|D}q^tdd|D}n&t|trT|jjtdd}n |t}|tj tfS)NcSs g|]}t|st|ndqS)T)r+rq)rrrJrJrKr sz;GroupBy._bool_agg..objs_to_bool..cSsg|] }t|qSrJ)rq)rrrJrJrKr sF)r) r(r~r$r|r._datar&rqviewint8)r8)skipnarJrK objs_to_bools  z'GroupBy._bool_agg..objs_to_boolFz np.ndarrayrrq)r inferencenullabler\cSs.|rt|jtdd|dkS|j|ddSdS)NF)rr)r/r&rq)rr>r?rJrJrKresult_to_boolsz)GroupBy._bool_agg..result_to_boolZ group_any_allT) aggregater cython_dtype needs_values needs_maskneeds_nullablepre_processingpost_processingval_testr<)F)_get_cythonized_resultr~rr;)rIrHr<r=r@rJ)r<rK _bool_aggs zGroupBy._bool_aggrG)rU)r<cCs |d|S)a Return True if any value in the group is truthful, else False. Parameters ---------- skipna : bool, default True Flag to ignore nan values during truth testing. Returns ------- Series or DataFrame DataFrame or Series of boolean values, where a value is True if any element is True within its respective group, False otherwise. any)rJ)rIr<rJrJrKrK,sz GroupBy.anycCs |d|S)a Return True if all values in the group are truthful, else False. Parameters ---------- skipna : bool, default True Flag to ignore nan values during truth testing. Returns ------- Series or DataFrame DataFrame or Series of boolean values, where a value is True if all elements are True within its respective group, False otherwise. r)rJ)rIr<rJrJrKr@sz GroupBy.allcCstdS)z Compute count of group, excluding missing values. Returns ------- Series or DataFrame Count of values within each group. N)r)rIrJrJrKr1Ts z GroupBy.count)Zsee_also)rcs2||jdfddd}|j|jddS)a Compute mean of groups, excluding missing values. Parameters ---------- numeric_only : bool, default True Include only float, int, boolean columns. If None, will attempt to use everything, then use only numeric data. Returns ------- pandas.Series or pandas.DataFrame %(see_also)s Examples -------- >>> df = pd.DataFrame({'A': [1, 1, 2, 1, 2], ... 'B': [np.nan, 2, 3, 4, 5], ... 'C': [1, 2, 1, 1, 2]}, columns=['A', 'B', 'C']) Groupby one column and return the mean of the remaining columns in each group. >>> df.groupby('A').mean() B C A 1 3.0 1.333333 2 4.0 1.500000 Groupby two columns and return the mean of the remaining column. >>> df.groupby(['A', 'B']).mean() C A B 1 2.0 2.0 4.0 1.0 2 3.0 1.0 5.0 2.0 Groupby one column and return the mean of only particular column in the group. >>> df.groupby('A')['B'].mean() A 1 3.0 2 4.0 Name: B, dtype: float64 meancst|jdS)N)r)r>rL)r)rrJrKrzr{zGroupBy.mean..)rrrG)r)rrrrl)rIrrrJ)rrKrLbs 3  z GroupBy.meancs2||jdfddd}|j|jddS)a Compute median of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex Parameters ---------- numeric_only : bool, default True Include only float, int, boolean columns. If None, will attempt to use everything, then use only numeric data. Returns ------- Series or DataFrame Median of values within each group. mediancst|jdS)N)r)r>rM)r)rrJrKrzr{z GroupBy.median..)rrrG)r)rrrrl)rIrrrJ)rrKrMs   zGroupBy.medianr)ddofc Cs&|jdddddttjdd|dS)aw Compute standard deviation of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex. Parameters ---------- ddof : int, default 1 Degrees of freedom. Returns ------- Series or DataFrame Standard deviation of values within each group. Z group_varTcSs t|S)N)r~sqrt)r8r>rJrJrKrzr{zGroupBy.std..)rA needs_countsrCneeds_2drBrGrN)rIr~rfloat64)rIrNrJrJrKstds z GroupBy.stdc sVdkr,|tj}|jdfdd|dSfdd}t| ||SQRXdS)ac Compute variance of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex. Parameters ---------- ddof : int, default 1 Degrees of freedom. Returns ------- Series or DataFrame Variance of values within each group. rvarcst|jdS)N)rN)r>rT)r)rNrJrKrzr{zGroupBy.var..)rrcs |jdS)N)rN)rT)r)rNrJrKrzr{N)rrrrr_r )rIrNrrrJ)rNrKrTs   z GroupBy.varcCs|j|d}|jdkr*|t|}n`|j|j}|}|j |}|j |}|j dd|ft|j dd|f<|S)a Compute standard error of the mean of groups, excluding missing values. For multiple groupings, the result index will be a MultiIndex. Parameters ---------- ddof : int, default 1 Degrees of freedom. Returns ------- Series or DataFrame Standard error of the mean of values within each group. )rNrN) rSrr~rOr1rrrfuniqueZget_indexer_forr)rIrNrcolscountsZ result_ilocsZ count_ilocsrJrJrKsems    .z GroupBy.semcCsX|j}t|jjtr,|j||jjd}n ||}|jsJ| d }|j |ddS)z Compute group sizes. Returns ------- DataFrame or Series Number of rows in each group as a Series if as_index is True or a DataFrame if as_index is False. )rUsizer) fill_value) rgrY issubclassrlr r>r7rUrcrename reset_index_reindex_output)rIrrJrJrKrYs  z GroupBy.sizesum)fnamenoZmc)rrc CsF||}t|dd|j||dtjd}WdQRX|j|ddS)NrmTadd)rrrrr)rZ)rrr!rr~r_r^)rIrrrrJrJrKr_.s z GroupBy.sumprodcCs||}|j||dtjdS)Nrc)rrrr)rrr~rc)rIrrrJrJrKrcBs z GroupBy.prodmincCs|j||dtjdS)Nrd)rrrr)rr~rd)rIrrrJrJrKrdMsz GroupBy.minmaxcCs|j||dtjdS)Nre)rrrr)rr~re)rIrrrJrJrKreTsz GroupBy.maxfirstcCs$d ddddd}|j||d|dS) Nrrrp)rlrdcSsHdddd}t|tr&|j||dSt|tr8||Stt|dS)Nr>)rcSs&|jt|j}t|stjS|dS)z-Helper function for first item that isn't NA.r)r$r,rrr~r)rarrrJrJrKrf_sz2GroupBy.first..first_compat..first)rd)r|r5rRr>rr)rlrdrfrJrJrK first_compat^s   z#GroupBy.first..first_compatrf)rrrr)r)r)rIrrrhrJrJrKrf[s z GroupBy.firstlastcCs$d ddddd}|j||d|dS) Nrrrp)rlrdcSsHdddd}t|tr&|j||dSt|tr8||Stt|dS)Nr>)rcSs&|jt|j}t|stjS|dS)z,Helper function for last item that isn't NA.r)r$r,rrr~r)rrgrJrJrKrixsz/GroupBy.last..last_compat..last)rd)r|r5rRr>rr)rlrdrirJrJrK last_compatws   z!GroupBy.last..last_compatri)rrrr)r)r)rIrrrjrJrJrKrits z GroupBy.lastr5cCs~|jjdkrl|j}t|j}|s(td|jjd|jdddd}dd d d g}|jj ||jj |d }| |S| d d|j S)a Compute open, high, low and close values of a group, excluding missing values. For multiple groupings, the result index will be a MultiIndex Returns ------- DataFrame Open, high, low and close values within each group. rzNo numeric types to aggregaterAohlcrr)rdropenhighlowclose)rrcSs|S)N)rk)rrJrJrKrzr{zGroupBy.ohlc..)rlrrr'rr2rgZ_cython_operationrZ_constructor_expanddimrr^Z_apply_to_column_groupbysr)rIrlZ is_numericrZ agg_namesrrJrJrKrks    z GroupBy.ohlcc s>t|,|fdd}|jdkr,|jS|SQRXdS)Ncs |jfS)N)describe)r)rOrJrKrzr{z"GroupBy.describe..r)r_rRrdrZunstack)rIrOrrJ)rOrKrps   zGroupBy.describecOsddlm}|||f||S)a< Provide resampling when using a TimeGrouper. Given a grouper, the function resamples it according to a string "string" -> "frequency". See the :ref:`frequency aliases ` documentation for more details. Parameters ---------- rule : str or DateOffset The offset string or object representing target grouper conversion. *args, **kwargs Possible arguments are `how`, `fill_method`, `limit`, `kind` and `on`, and other arguments of `TimeGrouper`. Returns ------- Grouper Return a new grouper with our resampler appended. See Also -------- Grouper : Specify a frequency to resample with when grouping by a key. DatetimeIndex.resample : Frequency conversion and resampling of time series. Examples -------- >>> idx = pd.date_range('1/1/2000', periods=4, freq='T') >>> df = pd.DataFrame(data=4 * [range(2)], ... index=idx, ... columns=['a', 'b']) >>> df.iloc[2, 0] = 5 >>> df a b 2000-01-01 00:00:00 0 1 2000-01-01 00:01:00 0 1 2000-01-01 00:02:00 5 1 2000-01-01 00:03:00 0 1 Downsample the DataFrame into 3 minute bins and sum the values of the timestamps falling into a bin. >>> df.groupby('a').resample('3T').sum() a b a 0 2000-01-01 00:00:00 0 2 2000-01-01 00:03:00 0 1 5 2000-01-01 00:00:00 5 1 Upsample the series into 30 second bins. >>> df.groupby('a').resample('30S').sum() a b a 0 2000-01-01 00:00:00 0 1 2000-01-01 00:00:30 0 0 2000-01-01 00:01:00 0 1 2000-01-01 00:01:30 0 0 2000-01-01 00:02:00 0 0 2000-01-01 00:02:30 0 0 2000-01-01 00:03:00 0 1 5 2000-01-01 00:02:00 5 1 Resample by month. Values are assigned to the month of the period. >>> df.groupby('a').resample('M').sum() a b a 0 2000-01-31 0 3 5 2000-01-31 5 1 Downsample the series into 3 minute bins as above, but close the right side of the bin interval. >>> df.groupby('a').resample('3T', closed='right').sum() a b a 0 1999-12-31 23:57:00 0 1 2000-01-01 00:00:00 0 2 5 2000-01-01 00:00:00 5 1 Downsample the series into 3 minute bins and close the right side of the bin interval, but label each bin using the right edge instead of the left. >>> df.groupby('a').resample('3T', closed='right', label='right').sum() a b a 0 2000-01-01 00:00:00 0 1 2000-01-01 00:03:00 0 2 5 2000-01-01 00:03:00 5 1 r)get_resampler_for_grouping)Zpandas.core.resamplerq)rIrulerNrOrqrJrJrKresamplesb zGroupBy.resamplecOs,ddlm}||jf||j|jd|S)zV Return a rolling grouper, providing rolling functionality per group. r)RollingGroupby)_grouperZ _as_index)pandas.core.windowrtrrgrc)rIrNrOrtrJrJrKrolling s zGroupBy.rollingcOs(ddlm}||jf|d|ji|S)zc Return an expanding grouper, providing expanding functionality per group. r)ExpandingGroupbyru)rvrxrrg)rIrNrOrxrJrJrK expanding1s  zGroupBy.expandingcOs(ddlm}||jf|d|ji|S)zO Return an ewm grouper, providing ewm functionality per group. r)ExponentialMovingWindowGroupbyru)rvrzrrg)rIrNrOrzrJrJrKewmBs  z GroupBy.ewmzLiteral[('ffill', 'bfill')]) directionc Cs0|dkr d}|jdddttjd|||jdS)a  Shared function for `pad` and `backfill` to call Cython method. Parameters ---------- direction : {'ffill', 'bfill'} Direction passed to underlying Cython function. `bfill` will cause values to be filled backwards. `ffill` and any other values will default to a forward fill limit : int, default None Maximum number of consecutive values to fill. If `None`, this method will convert to -1 prior to passing to Cython Returns ------- `Series` or `DataFrame` with filled values See Also -------- pad : Returns Series with minimum number of char in object. backfill : Backward fill the missing values in the dataset. NrZgroup_fillna_indexerFT)rrDrBresult_is_indexr|limitre)rIr~rr#re)rIr|r~rJrJrK_fillRs z GroupBy._fillcCs|jd|dS)a6 Forward fill the values. Parameters ---------- limit : int, optional Limit of how many values to fill. Returns ------- Series or DataFrame Object with missing values filled. See Also -------- Series.pad: Returns Series with minimum number of char in object. DataFrame.pad: Object with missing values filled or None if inplace=True. Series.fillna: Fill NaN values of a Series. DataFrame.fillna: Fill NaN values of a DataFrame. ffill)r~)r)rIr~rJrJrKpadysz GroupBy.padcCs|jd|dS)a5 Backward fill the values. Parameters ---------- limit : int, optional Limit of how many values to fill. Returns ------- Series or DataFrame Object with missing values filled. See Also -------- Series.backfill : Backward fill the missing values in the dataset. DataFrame.backfill: Backward fill the missing values in the dataset. Series.fillna: Fill NaN values of a Series. DataFrame.fillna: Fill NaN values of a DataFrame. bfill)r~)r)rIr~rJrJrKbackfillszGroupBy.backfillzint | list[int]zLiteral[('any', 'all', None)])rrer\c Csntttf}t||tfs td|st|tr8|g}nt||rNtt|}tj|tjd}t |t | |}t |j ddd| }||B}|j j \} } } || dk@}|j|} |js| S|j j} | | || _|jst| tr| | } || } |jr| S| SQRXt||r0td|d krJtd |d |d krX|nd|} |jj||jd }|jdkr|jdkr|j j}|||j}n0d dlm }|||j|j|j|j|j!d\}} } |j"||j|jd}|#|$|}}|| kj%}t&|r&|'r&tj(|j)|<t&|jt&|ksPt&|t&|j jkr\|j j|_n||j j}|S)a Take the nth row from each group if n is an int, or a subset of rows if n is a list of ints. If dropna, will take the nth non-null row, dropna is either 'all' or 'any'; this is equivalent to calling dropna(how=dropna) before the groupby. Parameters ---------- n : int or list of ints A single nth value for the row or a list of nth values. dropna : {'any', 'all', None}, default None Apply the specified dropna operation before counting which row is the nth row. Returns ------- Series or DataFrame N-th value within each group. %(see_also)s Examples -------- >>> df = pd.DataFrame({'A': [1, 1, 2, 1, 2], ... 'B': [np.nan, 2, 3, 4, 5]}, columns=['A', 'B']) >>> g = df.groupby('A') >>> g.nth(0) B A 1 NaN 2 3.0 >>> g.nth(1) B A 1 2.0 2 5.0 >>> g.nth(-1) B A 1 4.0 2 5.0 >>> g.nth([0, 1]) B A 1 NaN 1 2.0 2 3.0 2 5.0 Specifying `dropna` allows count ignoring ``NaN`` >>> g.nth(0, dropna='any') B A 1 2.0 2 3.0 NaNs denote group exhausted when using dropna >>> g.nth(3, dropna='any') B A 1 NaN 2 NaN Specifying `as_index=False` in `groupby` keeps the original index. >>> df.groupby('A', as_index=False).nth(1) A B 1 1 2.0 4 2 5.0 z0n needs to be an int or a list/set/tuple of ints)rF)r+rrNz8dropna option with a list of nth values is not supported)rKrzQFor a DataFrame groupby, dropna must be either None, 'any' or 'all', (was passed z).r)rrd)r)ryrdrjrnrk)rcrn)*setrrr|rprr~r$r/r_Zin1dr6rgrrrcrrrmr:rr^rnrrrlrerdrirjisinrrrkrGrYnthrrrrKrloc)rIrreZvalid_containersZ nth_valuesZ nth_arrayZ mask_leftZ mask_rightr)rrr4rmax_lenZdroppedrdrgrZgrbsizesrrJrJrKrsjO               z GroupBy.nth?linear) interpolationc sFddlm}ddddddd dd fd d t|r`jd ddddttj|d Sfdd|D}||j|d}tt d|j jj dg}t |j jj }tt||j j_ t|tr||}n|j|jd}|||j j_ t|jjt|jgj}|j|jdSdS)a Return group values at the given quantile, a la numpy.percentile. Parameters ---------- q : float or array-like, default 0.5 (50% quantile) Value(s) between 0 and 1 providing the quantile(s) to compute. interpolation : {'linear', 'lower', 'higher', 'midpoint', 'nearest'} Method to use when the desired quantile falls between two points. Returns ------- Series or DataFrame Return type determined by caller of GroupBy object. See Also -------- Series.quantile : Similar method for Series. DataFrame.quantile : Similar method for DataFrame. numpy.percentile : NumPy method to compute qth percentile. Examples -------- >>> df = pd.DataFrame([ ... ['a', 1], ['a', 2], ['a', 3], ... ['b', 1], ['b', 3], ['b', 5] ... ], columns=['key', 'val']) >>> df.groupby('key').quantile() val key a 2.0 b 3.0 r)rrz"tuple[np.ndarray, np.dtype | None])r8r\cSst|rtdd}t|jrLt|tr:|jttj d}n|}ttj }nt |jrrt|trr|jttj d}nt |jrtd}t |t}n`t|jrtd}t |t}n:t|trt|rttj}|jttj d}n t |}||fS)Nz7'quantile' cannot be performed against 'object' dtypes!)rZna_valuezdatetime64[ns]ztimedelta64[ns])r(rr&rr|r1rfloatr~rr#r#r$Zasarrayr&r*r%rR)r8r>r4rJrJrK pre_processor s,        z'GroupBy.quantile..pre_processorz np.ndarrayz type | None)r8r>r\cs"|rt|rdks||}|S)N>midpointr)r&r&)r8r>)rrJrKpost_processor s  z(GroupBy.quantile..post_processorgroup_quantileTF) rArrCrDrBrFrGqrc s0g|](}jddddttj|d qS)rT)rArCrDrBrFrGrr)rIr~rrR)rZqi)rrrrIrJrKr s z$GroupBy.quantile..)rdrir)rdN)Zpandasrr)rIr~rrRrdrrrZnlevelsr$rr0rrr|r>Zreorder_levelsrreshaperwrflattenr) rIrrrresultsrorderZ index_namesrxrJ)rrrrIrKquantile_ s8#      zGroupBy.quantile)r+c CsLt|:|jj}|j|jjd|tjd}|s>|jd|}|SQRXdS)a) Number each group from 0 to the number of groups - 1. This is the enumerative complement of cumcount. Note that the numbers given to the groups match the order in which the groups would be seen when iterating over the groupby object, not the order they are first observed. Parameters ---------- ascending : bool, default True If False, number in reverse, from number of group - 1 to 0. Returns ------- Series Unique numbers for each group. See Also -------- .cumcount : Number the rows in each group. Examples -------- >>> df = pd.DataFrame({"A": list("aaabba")}) >>> df A 0 a 1 a 2 a 3 b 4 b 5 a >>> df.groupby('A').ngroup() 0 0 1 0 2 0 3 1 4 1 5 0 dtype: int64 >>> df.groupby('A').ngroup(ascending=False) 0 1 1 1 2 1 3 0 4 0 5 1 dtype: int64 >>> df.groupby(["A", [1,1,2,3,2,1]]).ngroup() 0 0 1 0 2 1 3 3 4 2 5 0 dtype: int64 r)rrN) r_rrr7rgrr~r#rw)rIr+rrrJrJrKngroup s= zGroupBy.ngroupc Cs:t|(|j|j}|j|d}|||SQRXdS)a Number each item in each group from 0 to the length of that group - 1. Essentially this is equivalent to .. code-block:: python self.apply(lambda x: pd.Series(np.arange(len(x)), x.index)) Parameters ---------- ascending : bool, default True If False, number in reverse, from length of group - 1 to 0. Returns ------- Series Sequence number of each element within each group. See Also -------- .ngroup : Number the groups themselves. Examples -------- >>> df = pd.DataFrame([['a'], ['a'], ['a'], ['b'], ['b'], ['a']], ... columns=['A']) >>> df A 0 a 1 a 2 a 3 b 4 b 5 a >>> df.groupby('A').cumcount() 0 0 1 1 2 2 3 0 4 1 5 3 dtype: int64 >>> df.groupby('A').cumcount(ascending=False) 0 3 1 2 2 1 3 1 4 0 5 0 dtype: int64 )r+N)r_rrrdr6r7)rIr+rZ cumcountsrJrJrKcumcount- s7  zGroupBy.cumcountaveragekeep)rr+ na_optionpctrdcsb|dkrd}t|||||ddkrLdd<|fddS|jd d d S) a Provide the rank of values within each group. Parameters ---------- method : {'average', 'min', 'max', 'first', 'dense'}, default 'average' * average: average rank of group. * min: lowest rank in group. * max: highest rank in group. * first: ranks assigned in order they appear in the array. * dense: like 'min', but rank always increases by 1 between groups. ascending : bool, default True False for ranks by high (1) to low (N). na_option : {'keep', 'top', 'bottom'}, default 'keep' * keep: leave NA values where they are. * top: smallest rank if ascending. * bottom: smallest rank if descending. pct : bool, default False Compute percentage rank of data within each group. axis : int, default 0 The axis of the object over which to compute the rank. Returns ------- DataFrame with ranking of values within each group >rtopbottomz3na_option must be one of 'keep', 'top', or 'bottom') ties_methodr+rrrrrcs|jfddS)NF)rdr)rank)r)rdrOrJrKrz r{zGroupBy.rank..rF)rrd)r)rpoprRr)rIrr+rrrdrrJ)rdrOrKri s%z GroupBy.rankcs<td|ddgdkr0|fddS|jdS)zq Cumulative product for each group. Returns ------- Series or DataFrame cumprodrr<rcs|jfdiS)Nrd)r)r)rdrOrJrKrz r{z!GroupBy.cumprod..)r)nvvalidate_groupby_funcrRr)rIrdrNrOrJ)rdrOrKr s zGroupBy.cumprodcs<td|ddgdkr0|fddS|jdS)zm Cumulative sum for each group. Returns ------- Series or DataFrame r-rr<rcs|jfdiS)Nrd)r-)r)rdrOrJrKrz r{z GroupBy.cumsum..)r-)rrrRr)rIrdrNrOrJ)rdrOrKr- s zGroupBy.cumsumc s(dkr|fddS|jdddS)zm Cumulative min for each group. Returns ------- Series or DataFrame rcstj|S)N)r~minimum accumulate)r)rdrJrKrz r{z GroupBy.cummin..cumminF)r)rRr)rIrdrOrJ)rdrKr s zGroupBy.cumminc s(dkr|fddS|jdddS)zm Cumulative max for each group. Returns ------- Series or DataFrame rcstj|S)N)r~maximumr)r)rdrJrKrz r{z GroupBy.cummax..cummaxF)r)rRr)rIrdrOrJ)rdrKr s zGroupBy.cummaxznp.dtypez int | None) rrBrArrPrCrQrErrD needs_ngroupsr}c& Ks||}| r|rtd|r.t|s.td| rNt| sBtd|sNtd|j}|j\}}}i}tt|}d}xt|D]\}}|j }|j }|rt |j sq|r|}nt |}tj||d}|r|d}t||}d}|rtj|jtjd}t||}|r|} | ry| | \} }Wndtk r}!zDt|!}|d d}"tjd t|jd |"d |"d tddwWdd}!~!XYnX| j|dd} |r| d} t|| }t||}| dk rt|| }| rt|tj}#t||#}| rt||}|r0t |t!}$t||$d}|r0t||$d}|f||rJ|d}| r\t"#||}|rl|||}t$j%||d}%|||%<qW|s|dkrt||r|&|S|'|SdS)a Get result for Cythonized functions. Parameters ---------- how : str, Cythonized function name to be called cython_dtype : np.dtype Type of the array that will be modified by the Cython call. aggregate : bool, default False Whether the result should be aggregated to match the number of groups numeric_only : bool, default True Whether only numeric datatypes should be computed needs_counts : bool, default False Whether the counts should be a part of the Cython call needs_values : bool, default False Whether the values should be a part of the Cython call signature needs_2d : bool, default False Whether the values and result of the Cython call signature are 2-dimensional. min_count : int, default None When not None, min_count for the Cython call needs_mask : bool, default False Whether boolean mask needs to be part of the Cython call signature needs_ngroups : bool, default False Whether number of groups is part of the Cython call signature needs_nullable : bool, default False Whether a bool specifying if the input is nullable is part of the Cython call signature result_is_index : bool, default False Whether the result of the Cython operation is an index of values to be retrieved, instead of the actual values themselves pre_processing : function, default None Function to be applied to `values` prior to passing to Cython. Function should return a tuple where the first element is the values to be passed to Cython and the second element is an optional type which the values should be converted to after being returned by the Cython operation. This function is also responsible for raising a TypeError if the values have an invalid type. Raises if `needs_values` is False. post_processing : function, default None Function to be applied to result of Cython function. Should accept an array of values as the first argument and type inferences as its second argument, i.e. the signature should be (ndarray, Type). If `needs_nullable=True`, a third argument should be `nullable`, to allow for processing specific to nullable values. **kwargs : dict Extra arguments to be passed back to Cython funcs Returns ------- `Series` or `DataFrame` with filled values z6'result_is_index' and 'aggregate' cannot both be True!z%'post_processing' must be a callable!z$'pre_processing' must be a callable!z>Cannot use 'pre_processing' without specifying 'needs_values'!)r)rrNZgroup_zDropping invalid columns in .zQ is deprecated. In a future version, a TypeError will be raised. Before calling .z=, select only columns which should be valid for the function.r)rF)r)r?r)rr)(rrrrgrrV libgroupbyrrrUrr'rrrr~Zzerosrrrwr#rrTreplacerr rrQr r&r+r:Zuint8r|r.rrr7r rr)&rIrrBrArrPrCrQrErrDrr}rFrGrOrgrrrwrZ base_func error_msgr rlrUrZ result_szrrZ inferencesrWr8rZhowstrr)Z is_nullableryrJrJrKrI sJ                        zGroupBy._get_cythonized_resultcsNdk sdksts0|fddS|jddttjdddS) u Shift each group by periods observations. If freq is passed, the index will be increased using the periods and the freq. Parameters ---------- periods : int, default 1 Number of periods to shift. freq : str, optional Frequency string. axis : axis to shift, default 0 Shift direction. fill_value : optional The scalar value to use for newly introduced missing values. Returns ------- Series or DataFrame Object shifted within each group. See Also -------- Index.shift : Shift values of Index. tshift : Shift the time index, using the index’s frequency if available. Nrcs|S)N)shift)r)rdrZfreqperiodsrJrKrz r{zGroupBy.shift..Zgroup_shift_indexerFT)rrBrr}r)r+rRrIr~rr#)rIrrrdrZrJ)rdrZrrrKr s z GroupBy.shiftrc s|dk sdkr*|fddSdkr:ddt|d}|j|jj|jd}|j|jd}||d S) z Calculate pct_change of each value to previous entry in group. Returns ------- Series or DataFrame Percentage changes within each group. Nrcs|jdS)N)r fill_methodr~rrd) pct_change)r)rdrrr~rrJrKrz s z$GroupBy.pct_change..r)r~)rd)rrrdr)rRrVrGrgcodesrdr) rIrrr~rrdZfilledZfill_grpZshiftedrJ)rdrrr~rrKr s zGroupBy.pct_changecCs@|||k}|jdkr(|j|S|jjdd|fSdS)a Return first n rows of each group. Similar to ``.apply(lambda x: x.head(n))``, but it returns a subset of rows from the original DataFrame with original index and order preserved (``as_index`` flag is ignored). Does not work for negative values of `n`. Returns ------- Series or DataFrame %(see_also)s Examples -------- >>> df = pd.DataFrame([[1, 2], [1, 4], [5, 6]], ... columns=['A', 'B']) >>> df.groupby('A').head(1) A B 0 1 2 2 5 6 >>> df.groupby('A').head(-1) Empty DataFrame Columns: [A, B] Index: [] rN)r^r6rdrr)rIrr)rJrJrKhead s    z GroupBy.headcCsD||jdd|k}|jdkr,|j|S|jjdd|fSdS)a Return last n rows of each group. Similar to ``.apply(lambda x: x.tail(n))``, but it returns a subset of rows from the original DataFrame with original index and order preserved (``as_index`` flag is ignored). Does not work for negative values of `n`. Returns ------- Series or DataFrame %(see_also)s Examples -------- >>> df = pd.DataFrame([['a', 1], ['a', 2], ['b', 1], ['b', 2]], ... columns=['A', 'B']) >>> df.groupby('A').tail(1) A B 1 a 2 3 b 2 >>> df.groupby('A').tail(-1) Empty DataFrame Columns: [A, B] Index: [] F)r+rN)r^r6rdrr)rIrr)rJrJrKtail s   z GroupBy.tailr)rrZr\c Cs|jj}t|dkr|S|jr"|Stdd|Ds8|Sdd|D}tj||jjd\}}|j r|j |j |ddd |i}|j f|Sd dt|D}t|\} } |jt| dd }||jjj |d|d }|j| d }|jddS)a If we have categorical groupers, then we might want to make sure that we have a fully re-indexed output to the levels. This means expanding the output space to accommodate all values in the cartesian product of our groups, regardless of whether they were observed in the data or not. This will expand the output space if there are missing groups. The method returns early without modifying the input if the number of groupings is less than 2, self.observed == True or none of the groupers are categorical. Parameters ---------- output : Series or DataFrame Object resulting from grouping and applying an operation. fill_value : scalar, default np.NaN Value to use for unobserved categories if self.observed is False. Returns ------- Series or DataFrame Object (potentially) re-indexed to include all possible groups. rcss|]}t|jttfVqdS)N)r|Zgrouping_vectorr0r:)rpingrJrJrKrV sz*GroupBy._reindex_output..cSsg|] }|jqSrJ)Z group_index)rrrJrJrKr[ sz+GroupBy._reindex_output..)rrFrZcss"|]\}}|jr||jfVqdS)N)rrU)rirrJrJrKrt s)labelsrd)rrZ)rjT)drop)rgrrrrmrKr<Z from_productrZ sortlevelrcrlZ_get_axis_namerdrrrrrZ set_indexrr]) rIrrZrZ levels_listrrdZ in_axis_grpsZg_numsZg_namesrJrJrKr^0 s0      zGroupBy._reindex_outputz float | NonezSequence | Series | None)rfracrweightsc sddlm}dk r>t|jjdfdd|jD}n dg|j}dk r\t |j |j|j }fddt ||D} || |j dS) a" Return a random sample of items from each group. You can use `random_state` for reproducibility. .. versionadded:: 1.1.0 Parameters ---------- n : int, optional Number of items to return for each group. Cannot be used with `frac` and must be no larger than the smallest group unless `replace` is True. Default is one if `frac` is None. frac : float, optional Fraction of items to return. Cannot be used with `n`. replace : bool, default False Allow or disallow sampling of the same row more than once. weights : list-like, optional Default None results in equal probability weighting. If passed a list-like then values must have the same length as the underlying DataFrame or Series object and will be used as sampling probabilities after normalization within each group. Values must be non-negative with at least one positive element within each group. random_state : int, array-like, BitGenerator, np.random.RandomState, optional If int, array-like, or BitGenerator (NumPy>=1.17), seed for random number generator If np.random.RandomState, use as numpy RandomState object. Returns ------- Series or DataFrame A new object of same type as caller containing items randomly sampled within each group from the caller object. See Also -------- DataFrame.sample: Generate random samples from a DataFrame object. numpy.random.choice: Generate a random sample from a given 1-D numpy array. Examples -------- >>> df = pd.DataFrame( ... {"a": ["red"] * 2 + ["blue"] * 2 + ["black"] * 2, "b": range(6)} ... ) >>> df a b 0 red 0 1 red 1 2 blue 2 3 blue 3 4 black 4 5 black 5 Select one row at random for each distinct value in column a. The `random_state` argument can be used to guarantee reproducibility: >>> df.groupby("a").sample(n=1, random_state=1) a b 4 black 4 2 blue 2 1 red 1 Set `frac` to sample fixed proportions rather than counts: >>> df.groupby("a")["b"].sample(frac=0.5, random_state=2) 5 5 2 2 0 0 Name: b, dtype: int64 Control sample probabilities within groups by setting weights: >>> df.groupby("a").sample( ... n=1, ... weights=[1, 1, 1, 0, 0, 1], ... random_state=1, ... ) a b 5 black 5 2 blue 2 0 red 0 r)rN)rcsg|]}j|qSrJ)r)rr )rrJrKr sz"GroupBy.sample..c s(g|] \\}}}|j|dqS))rrrr random_state)sample)rrrlw)rrrrrJrKr s)rd)rrr>rrrxrrwrrrgrrdr) rIrrrrrrwsZgroup_iteratorZsamplesrJ)rrrrrrKr s]   zGroupBy.sample) NrNNNNTTTFFFT)F)F)Tr)r)Tr)T)T)T)r)r)r)Fr)Fr)Fr)Fr)N)N)N)N)rr)T)T)rTrFr)r)r)r)r)rNrN)rrNNr)r)r)NNFNN)XrQrYrZr[rrrLrXrr]r^rrrrrrrrrrr _apply_docsformatrRrr rrrrr"r*r6rr7rJr _common_see_alsorKrr1rrrLrMrSrTrXrYr"_groupby_agg_method_templater_rcrdrerfrirkr5rprsrwryr{rrrrrrrrrrrr-rrrIrrrrr~NaNr^rrJrJrJrKrFs B,+ ) B*  #/ %)4 " , 9" f&-D:2  *'(##STFr6z_KeysArgType | Nonerpzops.BaseGrouper | Nonerq) rlbyrdrgrcrnrhrormrkrer\cCsjt|trddlm}|}n*t|tr8ddlm}|}ntd|||||||||||| | | | | dS)Nr) SeriesGroupBy)DataFrameGroupByzinvalid type: )rlrirdrjrgrfrrcrnrhrormrkre)r|r>Zpandas.core.groupby.genericrr5rr)rlrrdrjrgrfrrcrnrhrormrkrerrrrJrJrK get_groupby s,    r) NrNNNNTTTFFFT)xr[ __future__r contextlibrr} functoolsrrrtextwraprrtypingrrr r r r r rrrrrnumpyr~Zpandas._config.configrZ pandas._libsrrZpandas._libs.groupbyZ_libsrGrZpandas._typingrrrrrrrrZpandas.compat.numpyrrZ pandas.errorsrZpandas.util._decoratorsrr r!r"Zpandas.core.dtypes.commonr#r$r%r&r'r(r)r*Zpandas.core.dtypes.missingr+r,Z pandas.corer-Zpandas.core.algorithmscorerZpandas.core.arraysr.r/r0r1Zpandas.core.baser2r3r4Zpandas.core.commoncommonrZpandas.core.framer5Zpandas.core.genericr6Zpandas.core.groupbyr7r8r9Zpandas.core.indexes.apir:r;r<Zpandas.core.internals.blocksr=Zpandas.core.seriesr>Zpandas.core.sortingr?Zpandas.core.util.numba_r@rArBrrrrZ_transform_templateZ _agg_templaterEr_Z _KeysArgTyper`rrFrrJrJrJrKs   4  (  (        8+4|L   J