The irony is that the very mechanism that makes LLMs powerful during training (e.g. compressing raw data into compact, transferable representations) is exactly what we refuse to let them do after deployment.
这是一个极具洞察力的反直觉观点。文章指出，正是训练过程中使LLMs强大的压缩机制，在部署后却被我们拒绝使用。这暗示我们可能正在错失让AI真正进化的关键机会，同时也提出了一个重要问题：为什么我们不让AI在部署后继续学习？
What used to take a week of back-and-forth between briefs, mockups, and review rounds now happens in a single conversation.
这一断言将设计流程从一周缩短到一次对话，代表了一个极其反直觉的时间压缩。这种戏剧性的效率提升挑战了传统设计工作流程的基本假设，需要验证实际应用案例和可能的局限性。
a free model that matches GPT-4o and runs entirely on your phone
这一声明揭示了AI模型小型化和普及化的惊人速度，表明前沿AI技术从云端到移动设备的迁移只需23个月，这种压缩速度远超以往任何技术革命，将彻底改变AI的可用性和普及范围。
In 23 months, the same capability that needed 1.8 trillion parameters now fits in 4 billion parameters. A 450x compression.
令人惊讶的是：AI模型参数量在短短23个月内实现了450倍的压缩，这意味着原本需要超级计算机才能运行的强大AI模型现在可以完全在手机上运行。这种技术进步的速度远超摩尔定律，展示了算法优化和模型压缩技术的惊人突破。
a free model that matches GPT-4o and runs entirely on your phone
大多数人认为顶级AI模型需要庞大的计算资源和云端支持，但作者认为免费模型Gemma 4 E4B已经能在手机上完全运行并匹敌GPT-4o的性能，这打破了人们对AI模型大小和资源需求的固有认知。
31B QLoRA：22GB 显存可运行
令人惊讶的是：310亿参数的模型现在只需要22GB显存就能运行，这比传统方法节省了大量计算资源，使得在消费级硬件上运行大型语言模型成为可能， democratizing AI access。
The era of 1-bit LLMs is here — now with WebGPU acceleration!
令人惊讶的是：1位大语言模型时代的到来意味着每个参数只需1位存储空间，相比传统的32位浮点表示，这代表了模型压缩技术的重大突破，结合WebGPU加速，使AI计算效率提升数十倍。
reasoning models tend to produce much shorter reasoning traces (up to 50%) for the same problem under different context conditions compared to the traces produced when the problem is presented in isolation.
令人震惊的发现：同一道题，仅仅因为周围塞入了无关上下文，推理模型的思考链长度就缩短了最多 50%——而题目本身一字未改。这意味着我们以为在评估模型「解题能力」，实际上评估的是「在特定上下文包装下的解题能力」。所有在孤立问题上测得的推理 benchmark，都可能严重高估了模型在真实 Agent 场景中的实际推理深度。
TriAttention matches Full Attention reasoning accuracy while achieving 2.5x higher throughput or 10.7x KV memory reduction
大多数人认为在KV缓存压缩中，准确率和效率之间存在不可避免的权衡，但作者提出的TriAttention方法能够在保持全注意力推理准确度的同时，实现2.5倍的吞吐量提升或10.7倍的内存减少。这一结果挑战了当前领域内的效率-准确度权衡范式，表明可以通过创新方法打破这一传统限制。
queries rotate with position during RoPE, making representative queries very few, leading to poor top-key selection and unstable reasoning.
大多数人认为注意力机制中的查询(Query)向量在旋转位置编码(RoPE)后仍然具有足够的代表性来准确估计键(Key)的重要性，但作者认为这种旋转实际上导致代表性查询向量非常少，从而严重影响键值选择和推理稳定性。这一发现挑战了当前主流的KV缓存压缩方法的基础假设。
we use the distance preference characterized by these centers to score keys according to their positions, and also leverage Q/K norms as an additional signal for importance estimation
大多数人认为KV缓存压缩主要基于注意力分数或内容相似性，但作者提出使用向量中心决定的距离偏好和Q/K范数作为重要性估计的信号。这一方法将注意力机制从传统的基于内容相似性转向基于几何特征，是一种全新的压缩思路。
TriAttention matches Full Attention reasoning accuracy while achieving 2.5x higher throughput or 10.7x KV memory reduction
大多数人认为在大幅压缩KV缓存时必然会牺牲模型推理的准确性，但作者声称TriAttention在实现10.7倍内存减少的同时，仍能保持与完整注意力相同的推理准确性。这一结果挑战了业界在KV压缩与准确性之间的权衡认知。
Treatment of superficial vein reflux (see Varicose Veins, above) has been shown to decrease the recurrence rate of venous ulcers. Where there is substantial obstruction of the femoral or popliteal deep venous system, superficial varicosities supply the venous return and should not be removed.
Failure of venous insufficiency ulcerations to heal is most often due to inconsistent use of first-line treatment methods. Ongoing control of edema is essential to prevent recurrent ulceration; the use of compression stockings following ulcer healing is critical, with recurrence rates 2–20 times higher if compression stockings are not used
there's a potential for overly simple terms and symbols to feel so packed with meaning that it seems like that's a solution to the meeting crisis
for - language - compression - solution to meaning crisis?
improvements to PDF processing are necessary to improve the user experiencefor augmented reading interfaces
each venue, journal, source/publisher etc. having their own formatting preferences.
Dockray scan and the export
the “clothed” eye of inscription devices
!!! / Vertesi; Cahill; Mars too.
convolution (which is the underlying operation of a Gaussian blur) has to loop over each field of the Gaussian kernel for each pixel in the image. However, if you convert both the image as well as the Gaussian kernel to the frequency domain (using one of the many Fast Fourier Transform algorithms), convolution becomes an element-wise multiplication.
cf RMO compression studies? sublimation was it?
“seeing ma-chine,”
Agre, two models of privacy
opacity:aseachvariablesuchasapixeliscomputedasavectorinrelationtoallothervariables,
a new orientation to this word "opacity" openness transparency
Image of noise
cf Earl Sweatshirt album cover art: the blur and the "Gen Z hard cut" of video v. millenial pause, boomer angle (had shared in previous IG story, cant remember when)
informality, disposability are "gen ai imagery" really "disposable aesthetics"?
editing video in a language that one is only partially fluent in.
how does the translation work? context, agency
the idea behind literate programming, too -- the idea that the stuff for humans should be the default context, and the highly constrained stuff parsed by the computer should be an exceptional mode within that
try SoundCloud for notetaking — since Reduct already biases towards recognition
you're going to push people to make sound in the ways that are legible to the computer. you may start to think that the text is the 'canonical' format
legibility
a cheap physical object, made of sound instead of writing, where both consuming it and creating it feel immediately at hand.
audio being linearly experienced?
cf. Ted Chiang story of your life / Arrival alien language
the device immediately (or even in real time, while you're speaking?) prints out a little receipt that 'contains' (that is) the audio that it just recorded2
closest approximation to orthographic recording, almost: receipt printer formulation suggests something more akin to photography
very highly rendered fake world
Reportage über die the yBlob genannte Hitzewelle in Kalifornien. Vieles spricht dafür, dass die für damals außergewöhnliche marine Hizewelle Vorläuferin eines neuen Normalzustands war https://www.nytimes.com/2024/12/01/climate/california-farallones-ocean-warming.html
This would exploit the compressive strength of stone, which can be greater than that of concrete, combined with post-tensioning by stainless steel rods.
for - sustainable building - stone age 2.0 - stone for compression - post-tensioned steel rods for tension
lossy compression: drop some of your data in a way that doesn’t impact your final results too much.
If parts of your data don’t impact your analysis, no need to waste memory keeping extraneous details around.
I used to use TimeSnapper for that. The classic version is free.It did use a crapload of disk space though (20GB per week?), and most of the data is almost identical, so I started designing an algorithm to store only the differences between images before realizing I had reinvented video codecs... so I just made a ffmpeg one liner to convert the image sequences to mp4 :)
An interesting story, but also very inspiring to me.
一个有趣的故事，同时也对我很有启发。
Current ruby releases generate *.tar.gz, *.tar.bz2, *.tar.xz, and *.zip. But I think we can stop generating *.tar.bz2. I think *.tar.bz2 are less merit. For better size, *.tar.xz exist. For better compatibility, *.tar.gz and *.zip exist.
Imagine a large population of people living, seeing, learning, doing and generally going about their lives. As they do so, they accumulate beliefs. Depending on how smart they are, they also compress beliefs via abstraction, metaphor, subconscious pattern-recognition circuits, muscle memory, ritual, making and consuming art, going p-value fishing, exploring tantric sex, generating irreproducible peer-reviewed Science! and so on.
Compression of knowledge through abstractions ~ mental models.
compressed public keypk2={bi t,(t1,t2,t3)∈(Z2e2)3,A∈Fp2,ent_bi t,r∈Z256}
You should construct evergreen (permanent) notes based on concepts, not related to a source (e.g. a book) or an author.
Your mental models are compression functions. You make them more powerful by trying to use them on new information. Are you able to compress the new information with an already acquired function? Yes, then you've discovered an analogous concept across two different sources. Sort of? Then maybe there's an important difference, or maybe it's a clue that your compression function needs updating. And finally, no? Then perhaps this is an indication that you need to construct a new mental model – a new compression function.
The final stages of this sequence are caused by blood accumulation that forces the temporal lobe medially, with resultant compression of the third cranial nerve and eventually the brain stem.
Rethinking floating point for deep learning
【网络的压缩加速问题】
Facebook人工智能研究院的Jeff Johnson改进了一种新颖的浮点数表示法（posit），使其更加适用于神经网络的训练和推理，并在FPGA上进行了对比实验。和IEEE-754浮点数标准相比，本论文基于改进的浮点数系统，可以实现低bit神经网络训练和高效推理，不再需要后续的量化压缩过程就可以部署在嵌入式等资源受限终端。该论文提出的方法区别于神经网络模型的剪枝、量化等常规思路，直接从浮点数表示这个更加基本、底层的角度尝试解决模型的压缩加速问题，是一个很新颖的方式，且效果不错，值得深入研究。除了论文，作者还给出了代码实现和博客文章，帮助理解。
Graph 2 shows the trend for the diastolic pressure.
Post-exercise, diastolic pressure is higher than baseline; compression garments exaggerate this effect
Graph 1 shows the systolic pressure data (x-axis in mmHg). It can be noted that the test subjects start from an equal baseline condition, but after performing the swimming test the athletes not wearing the costume in the first control (20-30 mins) have an average systolic pressure that has dropped to approximately 90 mmHg.
Thus, compression garments may prevent circumstantial hypotension. In this case, one possibility is that the compression is delivering blood to the heart that would otherwise be shunted to the skin for heat dissipation.
Major societal transformations are linked to information and communication technologies, giving rise to processes of growing global interdependence. They in turn generate the approxi-mation of coevalness, the illusion of simultaneity by being able to link instantly people and places around the globe. Many other processes are also accelerated. Speed and mobility are thus gaining in momentum, leading in turn to further speeding up processes that interlink the move-ment of people, information, ideas and goods.
Evokes Virilio theories and social/political critiques on speed/compression, as cited by Adam (2004).
Also Hassan's work, also cited by Adam (2004).
Both studies reveal a positive correlation between polychronicity and speed values: The more polychronic the organization, the more doing things rapidly is valued in its culture. Although these consistent findings about the speed-polychronicity relationship support the explanation of the size- polychronicity relationship developed in this discussion, they are not a direct test of this explanation, which is, admittedly, speculative. More direct tests must await studies deliberately designed to investigate this explanation
Larger firms appear to more polychronic. That finding seems to follow Bluedorn's own speculative findings of a relationship between polychronic organizations and a culture that values speed (time compression).
Note: Organizational studies of polychronicity have been conducted through quantitative methods (surveys and questionnaires).
This industrial norm, as I suggested above, is fundamentally rooted in clock time and underpinned by naturalized assumptions about not just the capacity but also the need to commodify, compress and control time.
In Castells's analysis, time is not merely compressed but processed, and it is the network rather than acceleration that constitutes the discontinuity in a context of continuing compress10n.
compressed time vs processed time
acceleration (speed) vs network time
Virilio suggests that we can read the history of modernity as a series of innovations iu ever-increasing time compression. He argues that, through the ages, the wealth and power associated with ownership of land was equally tied to the capacity to traverse it and to the speed at which this could be achieved.
Cites French political theorist and technology critic Paul Virilio.
Virilio's engagement with speed integrates 3 concepts that evoke increasing tempos over 3 successive centuries: 19th century transport, 20th century transmission, and 21st century transplantion.
The concept of transplantation, which is more biological in origin/use, is not as broadly covered here as transport and transmission.
From the above we can see that Virilio understands human history in terms of a race with time, of ever-increasing speeds that transcend humans' biological capacity. To theorize culture without the dromosphere, that is, the sphere of beings in motion, he therefore sugges,ts, misses the key point of cultural activity and the uniqueness of the industrial way of life. Without an explicit conceptualization of the contemporary dromosphere -or in my terms timescape -it is thus difficult to fully understand the human-technology-science-economyequity-environmenr constellation. Moreover, it becomes impossible to appreciate that people are che weakest link when the time frames of action are compressed to zero and effects expand to eternity, when transmission and transplantation are instantaneous but their outcomes extend into an open future, when instantaneity and eternity are combined in a discordant fusion of all times.
Adam's critique of Virilio's incomplete theory on time compression as it related to cultural transformation. Claims it lacks adequate theoretical description/understanding of how people in the high-tempo dromosphere in his writings, (timescape in her work) interact with time.
Adam further notes how important it is to understand how people factor into discordant time compressions through everyday sociocultural interactions -- which she refers to as "the human-technology-science-economy-equity-environment constellation."
This is pretty dense theoretical work. Would help to find an example or two in the SBTF time study to make this idea a little more accessible.
the potential capacity of exterrirorial beings to be everywhere at once and nowhere in particular is inescapably tied to operators that are bounded by their embodied temporal limits of terrestrial existence and sequential information processing. The actual capacity for parallel absorption of knowledge, therefore, is hugely disappointing. Equally, the electronic capacity to be now-here and no-where has brought the body to a standstill.
Adam's critique of transmission technologies allowing people to be "now-here and no-where" perhaps also helps unpacks some of the tensions for SBTF's global social coordination.
Could this be some of the unconscious motive to use terms that situate volunteers with one another as they attempt to grapple with tempo-imposed friction points which work against "terrestrial existence" and "sequential information processing"?
The overload of information, for example, is becoming so extensive that taking advantage of only the tiniest fraction of it not only blows apart the principle of instantaneity and 'real-time' communication, but also slows down operators to a pomt where they lose themselves in the eternity of electronically networked information.
High tempo Information overload exacerbates time compression and thus impacts temporal sensemaking through typical means via chronologies, linear information processing, and past/present/future contexts.
The intensive {electronic) present, Virilio suggests, is no longer part of chronological time; we have to conceptualize it instead _as chronoscopic time. Real space, he argues, is making room for decontextualized 'real-time' processes and intensity takes over from extensity.11 This in turn has consequences and, similar to the time compression in transport, the compression in transmission has led to a range of paradoxical effects.
Definition of chronoscopic time: While still bounded and defined by clock-time, like chronological time, chronoscopic experiences are more tempo-driven and focused on a hyper-present real-time. Chronological time is situated in movement across a timeline of past, present, future where history and temporal story narrative arcs.
See Purser (2000) for a dromological analysis of Virilio's work on chronoscopic- and real-time.
With respect to twentieth-century transmission Virilio has in mind the wireless telegraph, telephone, radio and subsequent developments in computer and satellite communication, which have once more changed the relationship between time and movement across space. Together, these innovations in transmission replaced succession and duration with seeming simultaneity and instantaneity. Duration has been compressed to zero and the present extended spatially to encircle the globe: it became a global present.
In the example of ICT advancements (radio, telegraph, computer, etc.), Adam describes a shift in tempo of a person's temporal experience due to real-time transmission capabilities.
Tempo experiences that are successive or have some duration quality are transformed into a perceived sense of instantaneous and simultaneous "real time" experience.
When a sociotemporal experience is lighting up friction points between time and space -- is this where tempo and timelines begin to get entangled?
Is the computer-mediated "movement" between time and space the inflection point where social coordination begins to break down? That we don't have enough time to process or make sense of CMC-delivered information?
In economic production, time compression has been achieved by a number of means: by increasing the activity within the same unit of tifYle (through machines and the intensification of labour), reorganizing the sequence and ordering of activities (Tay!orism and Fordism), using peaks and troughs more effective!�· (flexibilization), and by eliminating all unproductive times from the process ( the just-in-time system of production, delivery and consumption).
Time compression considers how time moves across space.
Valorizing speed (aka "time compression" per Marx and CUNY Anthropology and Geography professor David Harvey) is a political and economic goal of Western industrialized nations.
Speed also provides competitive advantages, whether for technological advancements, cultural movements and species biological evolution.
A third paradox is only hinted at by Vmho, when he suggests that conflict is to be expected between democracy and dromocracy, the politics that take account of time and the speed of movement across space.20 It concerns the sociopolitical and socioeconomic relations associated with advances in transport speed, which affect different indivi�uals, groups and classes of society in uneven ways.
Transportation speed is entangled with social equity and power: time-poor, cash-rich can "buy" time through labor, efficient technologies but the time-rich, cash-poor cannot trade time to become wealthy wealth.
Rifkin and Howard point out, 'the faster we speed up, the faster we degrade
Virilio writes that speed, or time compression, also contributes to adverse social and environmental impacts, per Rifkin and Howard: "the faster we speed up, the faster we degrade."
In the light of this evidence, which is fully supported by transport research, 17 Virilio formulated the �romological law, which states that increase in speed mcreases the potential for gridlock.
Virilio's dromological law: "increase in speed, increases the potential for gridlock."
This evokes environmental concerns as well as critiques of political privilege/power wrt to elites with access to fast transport options and those with less clout relegated to public transportation, traffic jams, less reliable options, etc.
the Reformation had a major role to play in the metamorphosis of time from God's gift to commodified, comp�essed, colonized and controlled resource. These four Cs of mdustrial time -comrnodification, compression, colonization and control -will be the focus in these pages, the fifth C of the creation of clock time having been discussed already in the previous chapter. I show their interdependence and id�ntify some of the socio-environmental impacts of those parttcular temporal relations.
Five C's of industrial time: Commodification, compression, colonialization, control, and clock time.
Lossless and lossy streaming
No streaming and no inter-frame compression - along the temporal axis... Just bz2, (google's) snappy and zlib...
It was far more important to focu s our efforts on compressing the depth data . Developing our own algorithm wa s outside the scope of this project, and existing work on depth data compression is not yet sufficiently developed to implement here, so we decided to work with popular and freely available lossless algorithms implemented in C++. We chose three separate algorithms with different performance characteristics. First, the bzip2 algorithm aims for maximum compression with slower speed. A second algorithm developed by Google, snappy , aim s for maximum speed with less compression. Finally the zlib algorithm aims for a middle ground between speed and compression. To test, we compress ed 500 different depth frames with each algorithm and calculate the mean speed and compression ratio for each.
Some cracks are more likely to happen than others, in extension, perpendicular to the force applied is where it will break first ( boof is the breaking sound apparently) in engineering it is most often extension. Geologists work in compression. What happens when you're pushing the original cracks together makes cracks in the vertical orientation will appear as the cracks in the horizontal orientation close.
How do we know? Milk in glass bottles- glass in refillable bottles but they would break easily and someone wondered why? so they discovered there were microcracks and would make the bottles break easier- not all glass is created equal.
Difference between XZ and LZMA2 Short answer: xz is a format that (currently) only uses the lzma2 compression algorithm. Long answer: think of xz as a container for the compression data generated by the lzma2 algorithm. We also have this paradigm for video files for example: avi/mkv/mov/mp4/ogv are containers, and xvid/x264/theora are compression algorithms. The confusion is often made because currently, the xz format only supports the lzma2 algorithm (and it’ll remain the default, even if some day, others algorithms may be added). This confusion doesn’t happen with other formats/algorithms, as for example gzip is both a compression algorithm and a format. To be exact, the gzip format only supports to encapsulate data generated by gzip… the compression algorithm. In this article I’ll use “xz” to say “the lzma2 algorithm whose data is being encapsulated by the xz format”. You’ll probably agree it’s way simpler
The key here is the notion of a format as a container. Lots of content is moving towards that notion-- that a "file" is really an opaque (to the OS filesystem) directory or container of some sort and some other program understands the format of the "file" as a container to know how to open it to access the files inside.