<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>information-theory on S Anand</title>
    <link>https://www.s-anand.net/blog/tag/information-theory/</link>
    <description>Recent content in information-theory on S Anand</description>
    <generator>Hugo -- 0.164.0</generator>
    <language>en-us</language>
    <lastBuildDate>Sun, 14 Jun 2026 00:00:00 +0000</lastBuildDate>
    <atom:link href="https://www.s-anand.net/blog/tag/information-theory/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Things I Learned - 14 Jun 2026</title>
      <link>https://www.s-anand.net/blog/things-i-learned-14-jun-2026/</link>
      <pubDate>Sun, 14 Jun 2026 00:00:00 +0000</pubDate>
      <guid>https://www.s-anand.net/blog/things-i-learned-14-jun-2026/</guid>
      <description>&lt;p&gt;This week, I learned:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Overheard a journalist saying: &amp;ldquo;I can tell when humans are lying. There are no tell tale signs of AI lying. At least &lt;em&gt;I&lt;/em&gt; don&amp;rsquo;t have any.&amp;rdquo;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://github.com/public-clis/rdt-cli&#34;&gt;rdt-cli&lt;/a&gt; is a Reddit CLI. It uses a clever trick: it auto-detects installed browsers and extracts cookies (supports Chrome, Firefox, Edge, Brave). So, if you&amp;rsquo;re logged into Reddit on any browser, &lt;code&gt;uvx --from rdt-cli rdt whoami&lt;/code&gt; automatically shows who you are logged in as. (The &lt;a href=&#34;https://github.com/public-clis/public-clis&#34;&gt;public-clis&lt;/a&gt; repo also lists other useful CLIs like &lt;a href=&#34;https://github.com/public-clis/twitter-cli&#34;&gt;twitter-cli&lt;/a&gt;, )&lt;/li&gt;
&lt;li&gt;Currently, a $20 Claude Pro gives you ~$400 and a $100 Claude Max gives you ~$2,000 of API usage. For ChatGPT, the numbers are ~$700 and $3,500. &lt;a href=&#34;https://x.com/SemiAnalysis_/status/2064815044085318040&#34;&gt;SemiAnalysis&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;When Fable 5 refuses to answer questions, here&amp;rsquo;s the message that appears: &amp;ldquo;Fable 5 has safety measures that flag messages on most cybersecurity or biology topics. They may flag safe, normal content as well. These measures let us bring you Mythos-level capability in other areas sooner, and we&amp;rsquo;re working to refine them. Send feedback or &lt;a href=&#34;https://support.claude.com/en/articles/15363606&#34;&gt;learn more&lt;/a&gt;.&amp;rdquo; I managed to trigger this once while researching an M&amp;amp;A acquisition target. Clicking on &amp;ldquo;Edit and retry with Fable 5&amp;rdquo; triggered Opus 5 again, twice.&lt;/li&gt;
&lt;li&gt;DNA codons (A, T, C, G) encode proteins in triplets. There are &lt;a href=&#34;https://en.wikipedia.org/wiki/DNA_and_RNA_codon_tables&#34;&gt;64 triplets that map to 20 amino acids&lt;/a&gt;. Some like Leucine, have 6 codons. Some like Methionine have only one. Why? When creating genes, there&amp;rsquo;s a wobble, sometimes, at the 3rd codon. THe mapping minimizes that impact: small errors map to similar proteins. The more common proteins have more codons. There&amp;rsquo;s a lot of fascinating information science going on here. &lt;a href=&#34;https://gemini.google.com/share/cfa70dcab30c&#34;&gt;Gemini&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;ChatGPT now shows a &amp;ldquo;Check in&amp;rdquo; button when it&amp;rsquo;s thinking. Clicking on that gives you a work-in-progress answer while it continues thinking. When done, it &lt;em&gt;replaces&lt;/em&gt; the WIP answer with the final answer. A useful feature!&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    <item>
      <title>Classic texts in computer science</title>
      <link>https://www.s-anand.net/blog/classic-texts-in-computer-science/</link>
      <pubDate>Wed, 21 Jun 2006 12:00:00 +0000</pubDate>
      <guid>https://www.s-anand.net/blog/classic-texts-in-computer-science/</guid>
      <description>&lt;p&gt;&lt;a href=&#34;http://www.zafar.se/bkz/wiki/view/43bafac8c8570f4f&#34;&gt;Classic texts in computer science&lt;/a&gt;. Worth reading for the sheer insight.&lt;/p&gt;
&lt;p&gt;Update: The link didn&amp;rsquo;t seem to work in Feb 2007. Here&amp;rsquo;s the list.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;http://www.spatial.maine.edu/~worboys/processes/hoare%20axiomatic.pdf&#34;&gt;An axiomatic basis for computer programming&lt;/a&gt; by C. A. R. Hoare&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.usingcsp.com/&#34;&gt;Communicating Sequential Processes (CSP)&lt;/a&gt; by C. A. R. Hoare&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://homepages.inf.ed.ac.uk/gdp/publications/cbn_cbv_lambda.pdf&#34;&gt;Call-by-name, call-by-value, and the lambda calculus&lt;/a&gt; by Gordon Plotkin&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;ftp://ftp.cs.cmu.edu/user/jcr/theotypestr.pdf&#34;&gt;Towards a theory of type structure&lt;/a&gt; by John C. Reynolds&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;ftp://ftp.cs.cmu.edu/user/jcr/defint.ps.gz&#34;&gt;Definitional interpreters for higher-order programming languages&lt;/a&gt; by John C. Reynolds&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.slac.stanford.edu/pubs/slacreports/slac-r-114.html&#34;&gt;An APL Machine 1970&lt;/a&gt; by Philip S. Abrams&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.pipeline.com/~hbaker1/&#34;&gt;Henry Baker&amp;rsquo;s Archive of Research Papers (many classic Lisp papers)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www-db.stanford.edu/pub/papers/google.pdf&#34;&gt;The Anatomy of a Large-Scale Hypertextual Web Search Engin&lt;/a&gt; by Sergey Brin and Lawrence Page&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www-inst.eecs.berkeley.edu/~maratb/readings/NoSilverBullet.html&#34;&gt;No Silver Bullet: Essence and Accidents of Software Engineering&lt;/a&gt; by Frederic P. Brooks, Jr.&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.unil.ch/webdav/site/ling/shared/ElementStatText/Shannon1948.pdf&#34;&gt;A Mathematical Theory of Communication&lt;/a&gt; by Claude Shannon&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.cs.ucla.edu/~jkong/research/security/shannon1949.pdf&#34;&gt;Communication Theory of Secrecy Systems&lt;/a&gt; by Claude Shannon&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.cs.ubc.ca/~murphyk/Bayes/Charniak_91.pdf&#34;&gt;Bayesian Networks without Tears&lt;/a&gt; by Eugene Charniak&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.stanford.edu/class/ee398a/resources/ziv:77-SDC.pdf&#34;&gt;A Universal Algorithm for Sequential Data Compression&lt;/a&gt; by Jacob Ziv and Abraham Lempel&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.cs.duke.edu/~junyang/cps216/papers/codd-1970.pdf&#34;&gt;A Relational Model of Data for Large Shared Data Banks 1970&lt;/a&gt; by Edgar F. Codd&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://compilers.iecc.com/crenshaw/&#34;&gt;Let&amp;rsquo;s Build a Compiler 1988-1995&lt;/a&gt; by Jack Crenshaw&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://gnowledge.sourceforge.net/damashek-ngrams.pdf&#34;&gt;Gauging Similarity via N-Grams: Language-Independent Sorting, Categorization, and Retrieval of Text&lt;/a&gt; by Marc Damashek&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.dreamsongs.com/WorseIsBetter.html&#34;&gt;Worse Is Better&lt;/a&gt; by Richard P. Gabriel&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.cs.berkeley.edu/~necula/cs263/handouts/hoarehints.pdf&#34;&gt;Hints on Programming Language Design&lt;/a&gt; by C.A.R. Hoare&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.math.chalmers.se/~rjmh/Papers/whyfp.pdf&#34;&gt;Why Functional Programming Matters&lt;/a&gt; by John Hughes&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.research.ibm.com/journal/rd/174/ibmrd1704F.pdf&#34;&gt;The Design of APL&lt;/a&gt; by Kenneth E. Iverson&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.metaobject.com/papers/Smallhistory.pdf&#34;&gt;The Early History Of Smalltalk&lt;/a&gt; by Alan Kay&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://fresh.homeunix.net/~luke/misc/knuth-turingaward.pdf&#34;&gt;Computer Programming as an Art&lt;/a&gt; by Donald E. Knuth&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.cs.utah.edu/~wilson/compilers/old/papers/p157-landin.pdf&#34;&gt;The next 700 programming languages&lt;/a&gt; by Peter J. Landin&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www-formal.stanford.edu/jmc/recursive.html&#34;&gt;Recursive Functions of Symbolic Expressions and their Computation by Machine (Part I) 1960&lt;/a&gt; by John McCarthy&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.cs.wisc.edu/~bolo/shipyard/4th_1970/4th_1970.html&#34;&gt;FORTH - A Language for Interactive Computing&lt;/a&gt; by Charles H.Moore&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.norvig.com/21-days.html&#34;&gt;Teach Yourself Programming in Ten Years 2001&lt;/a&gt; by Peter Norvig&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.nhplace.com/kent/PS/About-PS.html&#34;&gt;Parenthetically Speaking, a collection of essays from the 1990s&lt;/a&gt; by Kent M. Pitman&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;ftp://publications.ai.mit.edu/ai-publications/500-999/AITR-595.ps&#34;&gt;The Definition and Implementation of a Computer Language based on constraints&lt;/a&gt; by Guy Lewis Steele Jr.&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.cs.umbc.edu/331/resources/papers/gls-grow-lang.pdf&#34;&gt;Growing a Language&lt;/a&gt; by Guy Lewis Steele Jr.&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www-pu.informatik.uni-tuebingen.de/users/klaeren/epigrams.html&#34;&gt;Epigrams on Programming&lt;/a&gt; by Alan J. Perlis&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://www.cs.helsinki.fi/u/gionis/cc05/cook.pdf&#34;&gt;The Complexity of Theorem Proving Procedures&lt;/a&gt; by Stephen A. Cook&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://web.media.mit.edu/~minsky/papers/steps.html&#34;&gt;Steps Toward Artificial Intelligence&lt;/a&gt; by Marvin Minsky&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://library.readscheme.org/page1.html&#34;&gt;The Original &amp;lsquo;Lambda Papers&amp;rsquo;&lt;/a&gt; by Guy Steele and Gerald Sussman&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://theory.lcs.mit.edu/~rivest/rsapaper.pdf&#34;&gt;A Method for Obtaining Digital Signatures and Public-Key Cryptosystems&lt;/a&gt; by R.L. Rivest, A. Shamir, and L. Adleman&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://cm.bell-labs.com/cm/cs/who/dmr/cacm.html&#34;&gt;The UNIX Time-Sharing System&lt;/a&gt; by Dennis Ritchie and Ken Thompson&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id=&#34;comments&#34;&gt;Comments&lt;/h2&gt;
&lt;!-- wp-comments-start --&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;jawahar&lt;/strong&gt; &lt;em&gt;6 Feb 2007 5:52 am&lt;/em&gt;:
anand this link is not working !!&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;a href=&#34;https://maxviv.wordpress.com/2016/06/05/classic-texts-in-computer-science/&#34;&gt;Classic-texts-in-computer-science – maxviv&lt;/a&gt;&lt;/strong&gt; &lt;em&gt;5 Jun 2016 7:15 pm&lt;/em&gt; &lt;em&gt;(pingback)&lt;/em&gt;:
[…] Classic texts in computer science […]&lt;/li&gt;
&lt;/ul&gt;
&lt;!-- wp-comments-end --&gt;
</description>
    </item>
    <item>
      <title>Better information is not always beneficial</title>
      <link>https://www.s-anand.net/blog/better-information-is-not-always-beneficial/</link>
      <pubDate>Sat, 24 Sep 2005 12:00:00 +0000</pubDate>
      <guid>https://www.s-anand.net/blog/better-information-is-not-always-beneficial/</guid>
      <description>&lt;p&gt;&lt;a href=&#34;http://online.wsj.com/public/article/0,,SB112734060508547844-4Gx7dIqhluIyFxPiOh2mkB8mXrY_20060922,00.html?mod=blog&#34;&gt;Better information is not always beneficial&lt;/a&gt;.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;For $795, LegalMetric LLC will tell you which judges rule most swiftly and which tend to favor patent holders. For lawyer and client, this knowledge can be very valuable. But does it increase the chances that the judge will come to a just decision?&lt;/p&gt;
&lt;p&gt;It is the sort of information that Nobel laureate Kenneth Arrow labeled &amp;ldquo;socially useless but privately valuable.&amp;rdquo; It doesn&amp;rsquo;t help the economy produce more goods or services. It creates nothing of beauty or pleasure. It simply helps someone get a bigger slice of the pie. Sure, if the product helps win cases, then both sides will buy it &amp;ndash; just as both sides in high-stakes product-liability cases invest in jury-selection experts and software &amp;ndash; and neither will have an unfair advantage. But does that make the society better off?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;hr&gt;
&lt;h2 id=&#34;comments&#34;&gt;Comments&lt;/h2&gt;
&lt;!-- wp-comments-start --&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Arun&lt;/strong&gt; &lt;em&gt;24 Sep 2005 4:51 pm&lt;/em&gt;:
Interesting article. An internet bubble qualifies too, i presume? :-)&lt;/li&gt;
&lt;/ul&gt;
&lt;!-- wp-comments-end --&gt;
</description>
    </item>
    <item>
      <title>Limits of computing power</title>
      <link>https://www.s-anand.net/blog/limits-of-computing-power/</link>
      <pubDate>Wed, 04 Jul 2001 12:00:00 +0000</pubDate>
      <guid>https://www.s-anand.net/blog/limits-of-computing-power/</guid>
      <description>&lt;p&gt;A bizzarre calculation on the &lt;a href=&#34;http://arstechnica.com/wankerdesk/01q2/limits/limits-1.html&#34;&gt;limits of computing power&lt;/a&gt; using fundamentals of quantum mechanics.&lt;/p&gt;
</description>
    </item>
    <item>
      <title>Worlds information growth</title>
      <link>https://www.s-anand.net/blog/worlds-information-growth/</link>
      <pubDate>Wed, 01 Nov 2000 12:00:00 +0000</pubDate>
      <guid>https://www.s-anand.net/blog/worlds-information-growth/</guid>
      <description>&lt;p&gt;A &lt;a href=&#34;http://www.sims.berkeley.edu/how-much-info/&#34;&gt;study&lt;/a&gt; shows that the world is producing about 250MB of info per person per year. It means that, on average, &lt;strong&gt;each of us&lt;/strong&gt; is writing 50 times the size of the complete works of Shakespeare &lt;strong&gt;every year&lt;/strong&gt;! If you&amp;rsquo;re wondering how that&amp;rsquo;s possible, just wander around this web page :-)&lt;/p&gt;
</description>
    </item>
  </channel>
</rss>
