why is Information intimately related to entropy and energy
"Some of themost fundamental rules in physics, the laws of thermodynamics, for example, and the laws that tell how collections of atoms move in a chunk of matter-are deep down, actually laws about information." Shannons helped translate differential equations into a form the computer could understand and creating designs of electrical relays and flip-flo switches. Shannon created boolean logic using mathematics of manipulating 0s and 1s. Shannon uses 0s and 1s to measure the mass flow of information; he included compression algorithms into the model by exploiting redundancy in a given message. A question with N possible outcomes would need log N bits of information to distinquish between the information. Informtion encoded in 1s and 0s cand answer any question, so long as that question has a finite answer. Written language is a stream of finite symbols. Each symbol can be represented as a stream of bits. Bits are the universal medium of information. Five bits can be compressed into a one or two bits through a mapping rule. The rules make the string redundant. Shannon creates his channel capacity theorem to explain how much stuff can be sent over communication lines. "Information is intimately related to entropy and energy. The function Shannon derived was, roughly speaking, a measure of how unpredictable a string of bits is. the less predictable it is, the less able you are to generate the entire message from a smaller string of bits-in other words, the less redundant. The less redundancy a message has, the more information it can contain, so by measuring this unpredictability, Shannon hoped to be able to get at the information stored in the message." In the marbles in the box, the distribution of half the marbles on both the left and right side had the highest entropy and the distribution with all the marbles on either the left or right side had the lowest entropy. The entropy distribution of 1s ands 0s of symbols directly relates to the amount of information of the stream.[Learn More ...]
&Links
