Low Complexity Algorithm for Probability Density Estimation Applied in Big Data Analysis
|
10.5120/17696-8650 |
Smail Tigani, Mouhamed Ouzzif, Abderrahim Hasbi and Rachid Saadane. Article: Low Complexity Algorithm for Probability Density Estimation Applied in Big Data Analysis. International Journal of Computer Applications 101(7):1-5, September 2014. Full text available. BibTeX
@article{key:article,
author = {Smail Tigani and Mouhamed Ouzzif and Abderrahim Hasbi and Rachid Saadane},
title = {Article: Low Complexity Algorithm for Probability Density Estimation Applied in Big Data Analysis},
journal = {International Journal of Computer Applications},
year = {2014},
volume = {101},
number = {7},
pages = {1-5},
month = {September},
note = {Full text available}
}
Abstract
Running inference algorithms on a huge quantity of data knows some perturbations and looses performance. One of Big Data aims is the design of fast inference algorithms able to extract hidden information on a big quantity of data. This paper proposes a new low complexity algorithm for probability density estimation given partial observations. In order to reduce the complexity of the algorithm, a finite numerical data support is adopted in this work and observations are classified by frequencies to reduce there number without loosing significance. By frequency classification we mean, the mapping from the space containing all observed values to a space containing each observable value associated with its observation frequency. This approach relies on Lagrange interpolation for approximating the frequencies with a polynomial function and then build the probability density function. To prove the reliability of the approach, a simulation is done and results shows the convergence of discussed parameters to the expected values. Big Data field can benefit considerably from proposed approach to achieve density estimation algorithms goal with low cost.
References
- V. S. Kumar Samparthi, Harsh K. Verma, Outlier Detection of Data in Wireless Sensor Networks Using Kernel Density Estimation, International Journal of Computer Applications, Vol. 5, No. 7, August 2010.
- H. Cheol Cho, M. Sami Fadali and K. Soon Lee, Online Probability Density Estimation of Nonstationary Random Signal using Dynamic Bayesian Networks, International Journal of Control, Automation and Systems, Vol. 6, No. 1, pp. 109-118, February 2008.
- A. Assenza, M. Valle, M. Verleysen, A Comparative Study of Various Probabilty Density Estimaton Methods for Data Analysis, International Journal of Computational Intelligence Systems, Vol. 1, No. 2, 2008, pp 188-201.
- Adriano Z. Zambom and R. Dias, A Review of Kernel Density Estimation with Applications to Econometrics, International Econometric Review, Vol. 5, No. 1, 2013, pp20-42.
- A. Sinha and S. Gupta, Fast Estimation of Nonparametric Kernel Density Through PDDP, and its Application in Texture Synthesis, International Academic Conference 2008 Visions of Computer Science, Vol. 5, No. 1, 2008, pp. 225-236.
- L. Kamberi, T. Zenku, Intrpolation of Functions with Application Software, International Journal of Pure and Applied Mathematics, Vol. 73, No. 2, 2011, pp 219-225.
- B. Eckel, Thinking in C++, Vol. 1 Second Edition, January 13, 2000.
- H. Schildt, C++: The Complete Reference, Third Edition,.
- M. Jakel, Genetically Evolved Agents for Stock Price Prediction, International Journal of Inventive Engineering and Sciences, Vol. 10, No. 2, 2013, pp 21-35.
- L. Lebart, A. Morineay and M. Piron, Statistique Exploratoire Multidimensionnelle, ISBN 2 10 0028863, Dunod, Paris, 1995.