Entropy in Information Theory from Many Perspectives and Various Mathematical Models

Aqib Ali, Samreen Naeem, Sania Anam, Muhammad Munawar Ahmed

Abstract


Entropy is a nebulous scientific notion. This study explains entropy's progression from engineering thermodynamics to biological sciences and how it is used in numerous applications. Clausius invented entropy to quantify heat transport in thermodynamics. Boltzmann links a thermal system's microstates and macrostates. Many academics have expanded entropy's semantic and practical domains. Shannon invented information entropy. This idea solved engineering communications issues. Many economics scholars and businesses utilize this approach to analyze ion collision. Entropy's multifaceted knowledge would change numerous scientific domains and their applications. Entropy evaluates a dataset's irregularity and unpredictability. The estimated entropy value starts at 0. Entropy increases with dataset irregularity. Statistics, mathematics, and information theory employ a formula combining the sigma sign, logarithm, and probability. In this investigation, several formulae and conclusions provide the same entropy result. Experiments with the University of California Irvine (UCI) benchmark datasets yielded the same entropy findings.

Keywords


Information Theory, Thermodynamic Entropy, Information Entropy, Mathematical Model

Full Text:

PDF

References


Jepsen, J., Milanese, C., Puszkiel, J., Girella, A., Schiavo, B., Lozano, G. A., ... & Klassen, T. (2018). Fundamental material properties of the 2LiBH4-MgH2 reactive hydride composite for hydrogen storage:(I) Thermodynamic and heat transfer properties. Energies, 11(5), 1081.

Ramstead, M. J., Friston, K. J., & Hipólito, I. (2020). Is the free-energy principle a formal theory of semantics? From variational density dynamics to neural and phenotypic representations. Entropy, 22(8), 889.

Dharmaprani, D., Dykes, L., McGavigan, A. D., Kuklik, P., Pope, K., & Ganesan, A. N. (2018). Information theory and atrial fibrillation (AF): a review. Frontiers in Physiology, 9, 957.

Habibzadeh, H., Kaptan, C., Soyata, T., Kantarci, B., & Boukerche, A. (2019). Smart city system design: A comprehensive study of the application and data planes. ACM Computing Surveys (CSUR), 52(2), 1-38.

Bru, M. F., & Bru, B. (2018). Dice games. Statistical science, 33(2), 285-297.

Merghadi, A., Yunus, A. P., Dou, J., Whiteley, J., ThaiPham, B., Bui, D. T., ... & Abderrahmane, B. (2020). Machine learning methods for landslide susceptibility studies: A comparative overview of algorithm performance. Earth-Science Reviews, 207, 103225.

Baldovin, M., Iubini, S., Livi, R., & Vulpiani, A. (2021). Statistical mechanics of systems with negative temperature. Physics Reports, 923, 1-50.

Gappmair, W. (1999). Claude E. Shannon: The 50th anniversary of information theory. IEEE Communications Magazine, 37(4), 102-105.

Ahn, S., Couture, S. V., Cuzzocrea, A., Dam, K., Grasso, G. M., Leung, C. K., ... & Wodi, B. H. (2019, June). A fuzzy logic-based machine learning tool for supporting big data business analytics in complex artificial intelligence environments. In 2019 IEEE international conference on fuzzy systems (FUZZ-IEEE) (pp. 1-6). IEEE.

Omuya, E. O., Okeyo, G. O., & Kimwele, M. W. (2021). Feature selection for classification using principal component analysis and information gain. Expert Systems with Applications, 174, 114765.

Xie, Z., Ma, W., Ma, Y., Hu, Z., Sun, G., Han, Y., ... & Fan, Y. (2021). Decision tree-based detection of blowing snow events in the European Alps. Hydrology and Earth System Sciences, 25(7), 3783-3804.

Khalili-Damghani, K., Abdi, F., & Abolmakarem, S. (2018). Hybrid soft computing approach based on clustering, rule mining, and decision tree analysis for customer segmentation problem: Real case of customer-centric industries. Applied Soft Computing, 73, 816-828.

Ali, A.; Naeem, S.; Anam, S.; Zubair, M. Agile Software Development Processes Implementing Issues and Challenges with Scrum, in Proceedings of the MOL2NET'22, Conference on Molecular, Biomedical & Computational Sciences and Engineering, 8th ed., 1–31 January 2023, MDPI: Basel, Switzerland, doi:10.3390/mol2net-08-13907

Mohammadiun, S., Hu, G., Gharahbagh, A. A., Mirshahi, R., Li, J., Hewage, K., & Sadiq, R. (2021). Optimization of integrated fuzzy decision tree and regression models for selection of oil spill response method in the Arctic. Knowledge-Based Systems, 213, 106676.

Liu, Y., Safavi, T., Dighe, A., & Koutra, D. (2018). Graph summarization methods and applications: A survey. ACM computing surveys (CSUR), 51(3), 1-34.

Anderson, D. R. (2008). Information theory and entropy. Model based inference in the life sciences: A primer on evidence, 51-82.

Wenzel, R., & Van Quaquebeke, N. (2018). The double-edged sword of big data in organizational and management research: A review of opportunities and risks. Organizational Research Methods, 21(3), 548-591.

Ding, D., Savi, M., & Siracusa, D. (2020, April). Estimating logarithmic and exponential functions to track network traffic entropy in P4. In NOMS 2020-2020 IEEE/IFIP Network Operations and Management Symposium (pp. 1-9). IEEE.

Qi, R., Sun, Z., Lin, Z., Niu, P., Hao, W., Song, L., ... & Long, G. L. (2019). Implementation and security analysis of practical quantum secure direct communication. Light: Science & Applications, 8(1), 1-8.

Mallatt, J. (2021). A traditional scientific perspective on the integrated information theory of consciousness. Entropy, 23(6), 650.

Pavlos, G. P., Karakatsanis, L. P., Iliopoulos, A. C., Pavlos, E. G., & Tsonis, A. A. (2018). Non-extensive statistical mechanics: Overview of theory and applications in seismogenesis, climate, and space plasma. Advances in Nonlinear Geosciences, 465-495.

Zubair, M.; Ali, A.; Naeem, S.; Anam, S. Real-Time Highway Abnormality Detection Using an Image Processing Algorithm, in Proceedings of the MOL2NET'22, Conference on Molecular, Biomedical & Computational Sciences and Engineering, 8th ed., 1–31 January 2023, MDPI: Basel, Switzerland, doi:10.3390/mol2net-08-13922

Bahiraei, M., Heshmatian, S., & Keshavarzi, M. (2019). A decision-making based method to optimize energy efficiency of ecofriendly nanofluid flow inside a new heat sink enhanced with flow distributor. Powder Technology, 342, 85-98.

Ma, X., Sha, J., Wang, D., Yu, Y., Yang, Q., & Niu, X. (2018). Study on a prediction of P2P network loan default based on the machine learning LightGBM and XGboost algorithms according to different high dimensional data cleaning. Electronic Commerce Research and Applications, 31, 24-39.

Niedenzu, W., Mukherjee, V., Ghosh, A., Kofman, A. G., & Kurizki, G. (2018). Quantum engine efficiency bound beyond the second law of thermodynamics. Nature communications, 9(1), 1-13.

Starikov, E. B. (2021). How many laws has thermodynamics? What is the sense of the entropy notion? Implications for molecular physical chemistry. Monatshefte für Chemie-Chemical Monthly, 152(8), 871-879.

Bazai, S. U., Jang-Jaccard, J., & Alavizadeh, H. (2021). A novel hybrid approach for multi-dimensional data anonymization for apache spark. ACM Transactions on Privacy and Security, 25(1), 1-25.

Bazai, S. U., Jang-Jaccard, J., & Alavizadeh, H. (2021). Scalable, high-performance, and generalized subtree data anonymization approach for Apache Spark. Electronics, 10(5), 589.

Bazai, S. U., & Jang-Jaccard, J. (2020). In-memory data anonymization using scalable and high performance rdd design. Electronics, 9(10), 1732.

Bazai, S. U., Jang-Jaccard, J., & Wang, R. (2017, December). Anonymizing k-NN classification on MapReduce. In International Conference on Mobile Networks and Management (pp. 364-377). Springer, Cham.

Bazai, S. U., & Jang-Jaccard, J. (2019, December). SparkDA: RDD-based high-performance data anonymization technique for Spark platform. In International Conference on Network and System Security (pp. 646-662). Springer, Cham.

Naeem, S., & Ali, A. (2022). Bees Algorithm Based Solution of Non-Convex Dynamic Power Dispatch Issues in Thermal Units. Journal of Applied and Emerging Sciences, 12(1).

Ali, A., & Naeem, S. (2022). The Controller Parameter Optimization for Nonlinear Systems Using Particle Swarm Optimization and Genetic Algorithm. Journal of Applied and Emerging Sciences, 12(1).

McCormack, S. J., & Navrotsky, A. (2021). Thermodynamics of high entropy oxides. Acta Materialia, 202, 1-21.

Purvis, B., Mao, Y., & Robinson, D. (2019). Entropy and its application to urban systems. Entropy, 21(1), 56.

Feidt, M., & Costea, M. (2019). Progress in Carnot and Chambadal modeling of thermomechanical engine by considering entropy production and heat transfer entropy. Entropy, 21(12), 1232.

Popovic, M. (2019). Thermodynamic properties of microorganisms: determination and analysis of enthalpy, entropy, and Gibbs free energy of biomass, cells and colonies of 32 microorganism species. Heliyon, 5(6), e01950.

Widom, M. (2018). Modeling the structure and thermodynamics of high-entropy alloys. Journal of Materials Research, 33(19), 2881-2898.

Kostic, M. M. (2020). Energy: Physics. In Managing Air Quality and Energy Systems (pp. 135-158). CRC Press.

Kalies, G. (2021). A solution of the time paradox of physics. Zeitschrift für Physikalische Chemie, 235(7), 849-874.

Singhal, A., & Sharma, D. K. (2021, March). Keyword extraction using Renyi entropy: a statistical and domain independent method. In 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS) (Vol. 1, pp. 1970-1975). IEEE.

Scarfone, A. M. (2022). Boltzmann Configurational Entropy Revisited in the Framework of Generalized Statistical Mechanics. Entropy, 24(2), 140.

Wallace, D. (2020). Gravity, entropy, and cosmology: In search of clarity. The British Journal for the Philosophy of Science.

Loos, S. A., Hermann, S., & Klapp, S. H. (2021). Medium entropy reduction and instability in stochastic systems with distributed delay. Entropy, 23(6), 696.

Javaheri Javid, M. A. (2019). Aesthetic Automata: Synthesis and Simulation of Aesthetic Behaviour in Cellular Automata (Doctoral dissertation, Goldsmiths, University of London).

Xing, Y. Z., Fu, W. C., Liu, X. B., Lu, F. P., Zhang, H. F., & Zheng, Y. M. (2020). Sensitivity of the mean field dynamics within quantum molecular dynamics. Nuclear Physics A, 1004, 122034.

Smith, A. L. (2022). Structure-property relationships in actinide containing molten salts-a review: understanding and modelling the chemistry of nuclear fuel salts. Journal of Molecular Liquids, 119426.

Beck, J. (2021). Journal, January 14, 1959. PAJ: A Journal of Performance and Art, 43(2), 4-25.

Anjaria, K. (2020). Negation and entropy: Effectual knowledge management equipment for learning organizations. Expert Systems with Applications, 157, 113497.

Machado, F., Kahanamoku-Meyer, G. D., Else, D. V., Nayak, C., & Yao, N. Y. (2019). Exponentially slow heating in short and long-range interacting Floquet systems. Physical Review Research, 1(3), 033202.

Rose, J. N. (2018). Nested Levels of Organized Systems: A New Model of Multiple Nested Interacting Entropies that Result in the Production of Complexity. In Proceedings of the 62nd Annual Meeting of the ISSS-2018 Corvallis, OR, USA (Vol. 1, No. 1).

Cai, Z., & Zheng, X. (2018). A private and efficient mechanism for data uploading in smart cyber-physical systems. IEEE Transactions on Network Science and Engineering, 7(2), 766-775.

Riaz, A., Bobescu, E., Ramesh, K., & Ellahi, R. (2021). Entropy analysis for cilia-generated motion of Cu-blood flow of nanofluid in an annulus. Symmetry, 13(12), 2358.

Geng, Z., Zhang, Y., Li, C., Han, Y., Cui, Y., & Yu, B. (2020). Energy optimization and prediction modeling of petrochemical industries: An improved convolutional neural network based on cross-feature. Energy, 194, 116851.

Zubair, M.; Ali, A.; Naeem, S.; Anam, S. Video Streams for The Detection of Thrown Objects from Expressways, in Proceedings of the MOL2NET'22, Conference on Molecular, Biomedical & Computational Sciences and Engineering, 8th ed., 1–31 January 2023, MDPI: Basel, Switzerland, doi:10.3390/mol2net-08-13932

Zubair, M.; Ali, A.; Anam, S. A DDDAS-Based Impact Area Simulation Study of Highway Abnormalities, in Proceedings of the MOL2NET'22, Conference on Molecular, Biomedical & Computational Sciences and Engineering, 8th ed., 1–31 January 2023, MDPI: Basel, Switzerland, doi:10.3390/mol2net-08-13923




DOI: http://dx.doi.org/10.36785/jaes.122548

Creative Commons License
Journal of Applied and Emerging Sciences by BUITEMS is licensed under a Creative Commons Attribution 4.0 International License.
Based on a work at www.buitms.edu.pk.
Permissions beyond the scope of this license may be available at http://journal.buitms.edu.pk/j/index.php/bj

Contacts | Feedback
© 2002-2014 BUITEMS