B 0d@sddZddlZddlZddlmZddlZddlZddl m Z ddl m Z ddl mZddlmZmZdd lmZdd lmZmZdd lmZmZmZdd lmZmZeejj Z!d dZ"ddZ#ddZ$d7ddZ%ddZ&ddZ'ddZ(d8ddZ)d d!Z*d9d%d&Z+d:d'd(Z,d)d*Z-d;d,d-Z.dcCsv|dkrtdtd}t|d|j\}}|dk rX|dkrX|t||krXtd||dkrx|t||krtd}nd}|dkrt | |}t |}|| ||j |jdd } || ||j |jdd } tj| | d tj| | d | | fSt|||d \} } } t| } t| } t | d t| ddd f| ddd f<t | d t| d ddf| d ddf<xtd |D] }| dd|f| |ddf}}t|d t|d }}tt|d tt|d }}t|t|}}t|t|}}||||}}||krJ||}||}|}n||}||}|}t | ||}||| dd|f<||| |ddf<qWd | | |k<d | | |k<|dkrn|dkr| }|| | d k<|| | d k<n|dkr^t |}| }t|| t| | d kd| | d k<t|| t| | d kd| | d k<ntd|df| | fS)aAlgorithms for NMF initialization. Computes an initial guess for the non-negative rank k matrix approximation for X: X = WH. Parameters ---------- X : array-like of shape (n_samples, n_features) The data matrix to be decomposed. n_components : int The number of components desired in the approximation. init : {'random', 'nndsvd', 'nndsvda', 'nndsvdar'}, default=None Method used to initialize the procedure. Default: None. Valid options: - None: 'nndsvd' if n_components <= min(n_samples, n_features), otherwise 'random'. - 'random': non-negative random matrices, scaled with: sqrt(X.mean() / n_components) - 'nndsvd': Nonnegative Double Singular Value Decomposition (NNDSVD) initialization (better for sparseness) - 'nndsvda': NNDSVD with zeros filled with the average of X (better when sparsity is not desired) - 'nndsvdar': NNDSVD with zeros filled with small random values (generally faster, less accurate alternative to NNDSVDa for when sparsity is not desired) - 'custom': use custom matrices W and H eps : float, default=1e-6 Truncate all values less then this in output to zero. random_state : int, RandomState instance or None, default=None Used when ``init`` == 'nndsvdar' or 'random'. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. Returns ------- W : array-like of shape (n_samples, n_components) Initial guesses for solving X ~= WH. H : array-like of shape (n_components, n_features) Initial guesses for solving X ~= WH. References ---------- C. Boutsidis, E. Gallopoulos: SVD based initialization: A head start for nonnegative matrix factorization - Pattern Recognition, 2008 http://tinyurl.com/nndsvd rUzThe 'init' value, when 'init=None' and n_components is less than n_samples and n_features, will be changed from 'nndsvd' to 'nndsvda' in 1.1 (renaming of 0.26).NzNMF initializationrandomzLinit = '{}' can only be used when n_components <= min(n_samples, n_features)nndsvdF)copy)out) random_staterrnndsvdanndsvdardz3Invalid init parameter: got %r instead of one of %r)NrWrXr\r])warningsrU FutureWarningrrminrformatrrmeanr ZrandnZastypedtypeabsr Z zeros_liker-maximumminimumrlen)rr=initepsr[ n_samples n_featuresavgrngr/r.USVjryZx_pZy_pZx_nZy_nZx_p_nrmZy_p_nrmZx_n_nrmZy_n_nrmZm_pZm_nuvsigmaZlbdrrr_initialize_nmfs;    00"&       *, rwc Cs|jd}t|j|}t||} |dkrF|jdd|d|7<|dkrV| |8} |rf||} n t|} tj| tj d} t ||| | S)zHelper function for _fit_coordinate_descent. Update W to minimize the objective function, iterating once over all coordinates. By symmetry, to update H, one can call _update_coordinate_descent(X.T, Ht, W, ...). rgN)rd) rrrr'r Zflat permutationZarangeZasarrayZintpr) rr.HtZl1_regZl2_regshuffler[r=HHtXHtrxrrr_update_coordinate_descents    r}-C6?Tc  Cst|jdd} t|dd}t| }xtd|dD]}d}|t||| ||| |7}| rp|t|j| |||| |7}|dkr||}|dkrP| rtd|||||kr2| rtd |dPq2W|| j|fS) aCompute Non-negative Matrix Factorization (NMF) with Coordinate Descent The objective function is minimized with an alternating minimization of W and H. Each minimization is done with a cyclic (up to a permutation of the features) Coordinate Descent. Parameters ---------- X : array-like of shape (n_samples, n_features) Constant matrix. W : array-like of shape (n_samples, n_components) Initial guess for the solution. H : array-like of shape (n_components, n_features) Initial guess for the solution. tol : float, default=1e-4 Tolerance of the stopping condition. max_iter : int, default=200 Maximum number of iterations before timing out. l1_reg_W : float, default=0. L1 regularization parameter for W. l1_reg_H : float, default=0. L1 regularization parameter for H. l2_reg_W : float, default=0. L2 regularization parameter for W. l2_reg_H : float, default=0. L2 regularization parameter for H. update_H : bool, default=True Set to True, both W and H will be estimated from initial guesses. Set to False, only W will be estimated. verbose : int, default=0 The verbosity level. shuffle : bool, default=False If true, randomize the order of coordinates in the CD solver. random_state : int, RandomState instance or None, default=None Used to randomize the coordinates in the CD solver, when ``shuffle`` is set to ``True``. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. Returns ------- W : ndarray of shape (n_samples, n_components) Solution to the non-negative least squares problem. H : ndarray of shape (n_components, n_features) Solution to the non-negative least squares problem. n_iter : int The number of iterations done by the algorithm. References ---------- Cichocki, Andrzej, and Phan, Anh-Huy. "Fast local algorithms for large scale nonnegative matrix and tensor factorizations." IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009. C)ordercsr) accept_sparsergrz violation:zConverged at iteration)r r'r r-r}print)rr.r/tolmax_iterrIrJrKrLupdate_Hverboserzr[ryrnn_iterZ violationZviolation_initrrr_fit_coordinate_descents*U  rc CsR|dkrT| dkrt||j} | r&| } n| } |dkrDt||j}t||} nt|||} t|rx| j}|j}n(| }|}| }|ddkrt ||dk<|ddkrt ||dk<|dkrtj |||dn6|dkr|dC}|dC}||9}n||dC}||9}t| |j} |dkrJ|dkr6tj |dd }|tj ddf} nt|rt |j}xt|jdD]^}t||ddf|}|ddkrt ||dk<||dC}t||j||ddf<qrWn||dC}t||j}|} |dkr| |7} |dkr| ||} t | | dk<| | } | }|dkrF||C}|||| fS) z&Update W in Multiplicative Update NMF.rNg?rg@r)rZ)r )r r'rYrrr(r"r#r$r)divider*newaxisr9rr-)rr.r/rTrIrKgammaH_sumr{r|r numerator denominator WH_safe_XWH_safe_X_datar3r4ZWHHtr7WHidelta_Wrrr_multiplicative_update_wsl                "      rcCs$|dkr,t|j|}tj|j||g}nt|||} t|rP| j} |j} n(| } |} | } |ddkrxt | | dk<|ddkrt | | dk<|dkrtj | | | dn6|dkr| dC} | dC} | | 9} n| |dC} | | 9} t|j| }|dkr$tj |dd} d| | dk<| d d tj f}nt|rt|j}xt|jdD]^}t||d d |f}|ddkrt ||dk<||dC}t|j||d d |f<qLWn| |dC} t|j| }|}|dkr||7}|dkr|||}t ||dk<||}|}|dkr ||C}|S) z&Update H in Multiplicative Update NMF.rg?rg@r)rZr)r N)r r'rr%r&r(r"r#r$rYr)rr*rr9rr-r)rr.r/rTrJrLrrrrrr3r4ZW_sumZWtWHr7rdelta_Hrrr_multiplicative_update_hs`                "      rrNc  Cst} t|}|dkr&dd|} n|dkr`. verbose : int, default=0 The verbosity level. shuffle : bool, default=False If true, randomize the order of coordinates in the CD solver. Returns ------- W : ndarray of shape (n_samples, n_components) Solution to the non-negative least squares problem. H : ndarray of shape (n_components, n_features) Solution to the non-negative least squares problem. n_iter : int Actual number of iterations. Examples -------- >>> import numpy as np >>> X = np.array([[1,1], [2, 1], [3, 1.2], [4, 1], [5, 0.8], [6, 1]]) >>> from sklearn.decomposition import non_negative_factorization >>> W, H, n_iter = non_negative_factorization(X, n_components=2, ... init='random', random_state=0) References ---------- Cichocki, Andrzej, and P. H. A. N. Anh-Huy. "Fast local algorithms for large scale nonnegative matrix and tensor factorizations." IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009. Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, 23(9). )rcsc)rrd)r=rirrTrrr[rDrErFrGrrzrHT) assume_finite)r.r/rN)r rrfloat32NMFr_fit_transform)rr.r/r=rirrrTrrrDrErFrGrHr[rrzZestrrrrnon_negative_factorizationjs(V "rc@seZdZdZd#dddddddd d d d d dd ddZddZddZddZddZd$ddZ d%ddZ d&ddZ dd Z d!d"Z dS)'ra4Non-Negative Matrix Factorization (NMF). Find two non-negative matrices (W, H) whose product approximates the non- negative matrix X. This factorization can be used for example for dimensionality reduction, source separation or topic extraction. The objective function is: .. math:: 0.5 * ||X - WH||_{loss}^2 + alpha\_W * l1_{ratio} * n\_features * ||vec(W)||_1 + alpha\_H * l1_{ratio} * n\_samples * ||vec(H)||_1 + 0.5 * alpha\_W * (1 - l1_{ratio}) * n\_features * ||W||_{Fro}^2 + 0.5 * alpha\_H * (1 - l1_{ratio}) * n\_samples * ||H||_{Fro}^2 Where: :math:`||A||_{Fro}^2 = \sum_{i,j} A_{ij}^2` (Frobenius norm) :math:`||vec(A)||_1 = \sum_{i,j} abs(A_{ij})` (Elementwise L1 norm) The generic norm :math:`||X - WH||_{loss}` may represent the Frobenius norm or another supported beta-divergence loss. The choice between options is controlled by the `beta_loss` parameter. The regularization terms are scaled by `n_features` for `W` and by `n_samples` for `H` to keep their impact balanced with respect to one another and to the data fit term as independent as possible of the size `n_samples` of the training set. The objective function is minimized with an alternating minimization of W and H. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=None Number of components, if n_components is not set all features are kept. init : {'random', 'nndsvd', 'nndsvda', 'nndsvdar', 'custom'}, default=None Method used to initialize the procedure. Default: None. Valid options: - `None`: 'nndsvd' if n_components <= min(n_samples, n_features), otherwise random. - `'random'`: non-negative random matrices, scaled with: sqrt(X.mean() / n_components) - `'nndsvd'`: Nonnegative Double Singular Value Decomposition (NNDSVD) initialization (better for sparseness) - `'nndsvda'`: NNDSVD with zeros filled with the average of X (better when sparsity is not desired) - `'nndsvdar'` NNDSVD with zeros filled with small random values (generally faster, less accurate alternative to NNDSVDa for when sparsity is not desired) - `'custom'`: use custom matrices W and H solver : {'cd', 'mu'}, default='cd' Numerical solver to use: 'cd' is a Coordinate Descent solver. 'mu' is a Multiplicative Update solver. .. versionadded:: 0.17 Coordinate Descent solver. .. versionadded:: 0.19 Multiplicative Update solver. beta_loss : float or {'frobenius', 'kullback-leibler', 'itakura-saito'}, default='frobenius' Beta divergence to be minimized, measuring the distance between X and the dot product WH. Note that values different from 'frobenius' (or 2) and 'kullback-leibler' (or 1) lead to significantly slower fits. Note that for beta_loss <= 0 (or 'itakura-saito'), the input matrix X cannot contain zeros. Used only in 'mu' solver. .. versionadded:: 0.19 tol : float, default=1e-4 Tolerance of the stopping condition. max_iter : int, default=200 Maximum number of iterations before timing out. random_state : int, RandomState instance or None, default=None Used for initialisation (when ``init`` == 'nndsvdar' or 'random'), and in Coordinate Descent. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. alpha : float, default=0.0 Constant that multiplies the regularization terms. Set it to zero to have no regularization. When using `alpha` instead of `alpha_W` and `alpha_H`, the regularization terms are not scaled by the `n_features` (resp. `n_samples`) factors for `W` (resp. `H`). .. versionadded:: 0.17 *alpha* used in the Coordinate Descent solver. .. deprecated:: 1.0 The `alpha` parameter is deprecated in 1.0 and will be removed in 1.2. Use `alpha_W` and `alpha_H` instead. alpha_W : float, default=0.0 Constant that multiplies the regularization terms of `W`. Set it to zero (default) to have no regularization on `W`. .. versionadded:: 1.0 alpha_H : float or "same", default="same" Constant that multiplies the regularization terms of `H`. Set it to zero to have no regularization on `H`. If "same" (default), it takes the same value as `alpha_W`. .. versionadded:: 1.0 l1_ratio : float, default=0.0 The regularization mixing parameter, with 0 <= l1_ratio <= 1. For l1_ratio = 0 the penalty is an elementwise L2 penalty (aka Frobenius Norm). For l1_ratio = 1 it is an elementwise L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2. .. versionadded:: 0.17 Regularization parameter *l1_ratio* used in the Coordinate Descent solver. verbose : int, default=0 Whether to be verbose. shuffle : bool, default=False If true, randomize the order of coordinates in the CD solver. .. versionadded:: 0.17 *shuffle* parameter used in the Coordinate Descent solver. regularization : {'both', 'components', 'transformation', None}, default='both' Select whether the regularization affects the components (H), the transformation (W), both or none of them. .. versionadded:: 0.24 .. deprecated:: 1.0 The `regularization` parameter is deprecated in 1.0 and will be removed in 1.2. Use `alpha_W` and `alpha_H` instead. Attributes ---------- components_ : ndarray of shape (n_components, n_features) Factorization matrix, sometimes called 'dictionary'. n_components_ : int The number of components. It is same as the `n_components` parameter if it was given. Otherwise, it will be same as the number of features. reconstruction_err_ : float Frobenius norm of the matrix difference, or beta-divergence, between the training data ``X`` and the reconstructed data ``WH`` from the fitted model. n_iter_ : int Actual number of iterations. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- DictionaryLearning : Find a dictionary that sparsely encodes data. MiniBatchSparsePCA : Mini-batch Sparse Principal Components Analysis. PCA : Principal component analysis. SparseCoder : Find a sparse representation of data from a fixed, precomputed dictionary. SparsePCA : Sparse Principal Components Analysis. TruncatedSVD : Dimensionality reduction using truncated SVD. References ---------- Cichocki, Andrzej, and P. H. A. N. Anh-Huy. "Fast local algorithms for large scale nonnegative matrix and tensor factorizations." IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009. Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, 23(9). Examples -------- >>> import numpy as np >>> X = np.array([[1, 1], [2, 1], [3, 1.2], [4, 1], [5, 0.8], [6, 1]]) >>> from sklearn.decomposition import NMF >>> model = NMF(n_components=2, init='random', random_state=0) >>> W = model.fit_transform(X) >>> H = model.components_ NrUrrNg-C6?rrgr@rF) rirrTrrr[rDrErFrGrrzrHc CsX||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ | |_ ||_ dS)N)r=rirrTrrr[rDrErFrGrrzrH)selfr=rirrTrrr[rDrErFrGrrzrHrrr__init__2sz NMF.__init__cCsddiS)NZrequires_positive_XTr)rrrr _more_tagsSszNMF._more_tagscCs|j|_|jdkr|jd|_t|jtjr6|jdkrHtd|jdt|jtjr`|jdkrrtd|jdt|jtj r|jdkrtd|jdt |j |_ d}|j |krtd|j d ||j d kr|j d krtd |j d |j |j d kr|jdkrtdt|jdkrs        j " s _V  \