in reply to Cesar A. Hidalgo at https://x.com/realAnthonyDean/status/1844409919161684366

via Anthony Dean @realAnthonyDean

For most purposes having a huge amount of accuracy isn’t too important.

Instead of representing the values as floating numbers, we can represent them as percentages between 0 and 100. We’ll be down to two-digit accuracy, but again for many use cases that’s sufficient. Plus, if this is output from a model, those last few digits of “accuracy” are likely to be noise, they won’t actually tell us anything useful.

Whole percentages have the nice property that they can fit in a single byte, an int8—as opposed to float64, which uses eight bytes:

If parts of your data don’t impact your analysis, no need to waste memory keeping extraneous details around.

optimization-procrastination trap is related to shiny object syndrome - the idea of tweaking one's system constantly

perfect tool trap - guess what? there isn't one

See also the discussion (in the comments) about optimization-after-the-fact in http://akkartik.name/post/mu-2019-2

pointer alias 的 optimization block 怎么理解？

Definition of "Algospeak"

In order to get around algorithms that demote content in social media feeds, communities have coined new words or new meanings to existing words to communicate their sentiment.

This is affecting TikTok in particular because its algorithm is more heavy-handed in what users see. This is also causing people who want to be seen to tailor their content—their speech—to meet the algorithms needs. It is like search engine optimization for speech.

Article discovered via Cory Doctorow at The "algospeak" dialect

Why column storage is better query optimized

For how detailed and insightful his analysis was -- which didn't elaborate or even touch on his not understanding the reason for adding the semicolon -- it sure appeared like he knew what it was for. Otherwise, the whole issue would/should have been about how he didn't understand that, not on how to keep adding the semicolon but do so in a faster way!

Then again, this comment from 3 months afterwards, indicates he may not think they are even necessary: https://github.com/rails/sprockets/issues/388#issuecomment-252417741

Anyway, just in case he really didn't know, the comment shortly below partly answers the question:

Since the common problem with concatenating JavaScript files is the lack of semicolons, automatically adding one (that, like Sam said, will then be removed by the minifier if it's unnecessary) seems on the surface to be a perfectly fine speed optimization.

This also alludes to the problem: https://github.com/rails/sprockets/issues/388#issuecomment-257312994

But the explicit answer/explanation to this question still remains unspoken: because if you don't add them between concatenated files -- as I discovered just to day -- you will run into this error:

   (intermediate value)(...) is not a function
       at something.source.js:1

, apparently because when it concatenated those 2 files together, it tried to evaluate it as:

   ({
     // other.js
   })()
   (function() {
     // something.js
   })();

It makes sense that a ; is needed.

Matrajt, Laura, Julia Eaton, Tiffany Leung, und Elizabeth R. Brown. „Vaccine Optimization for COVID-19: Who to Vaccinate First?“ Science Advances 7, Nr. 6 (1. Februar 2020): eabf1374. https://doi.org/10.1126/sciadv.abf1374.

TB (terrific broth) consistently generated the greatest amount of plasmid DNA, in agreement with its ability to produce higher cell titers. The superiority of TB was primarily due to its high levels of yeast extract (24 g/L) and was independent of glycerol, a unique component of this broth. Interestingly, simply preparing LB with similarly high levels of yeast extract (LB24 broth) resulted in plasmid yields that were equivalent to those of TB.

If you consider using asynchronous to be an optimization, then this could be surprising.

See also: https://github.com/choojs/nanohtml#static-optimizations

(this person later recommends this library)

Burda, Z., Kotwica, M., & Malarz, K. (2020). Ageing of complex networks. Physical Review E, 102(4), 042302. https://doi.org/10.1103/PhysRevE.102.042302

Romanini, Daniele, Sune Lehmann, and Mikko Kivelä. ‘Privacy and Uniqueness of Neighborhoods in Social Networks’. ArXiv:2009.09973 [Physics], 21 September 2020. http://arxiv.org/abs/2009.09973.

Landgrave, M. (2020). How Do Legislators Value Constituent’s (Statistical) Lives? COVID-19, Partisanship, and Value of a Statistical Life Analysis. https://doi.org/10.31235/osf.io/n93w2

Optimal Unemployment Benefits in the Pandemic. COVID-19 and the Labor Market. (n.d.). IZA – Institute of Labor Economics. Retrieved August 1, 2020, from https://covid-19.iza.org/publications/dp13389/

Martin, G., Hanna, E., & Dingwall, R. (2020). Face masks for the public during Covid-19: An appeal for caution in policy [Preprint]. SocArXiv. https://doi.org/10.31235/osf.io/uyzxe

Sperrin, M., Martin, G. P., Sisk, R., & Peek, N. (2020). Missing data should be handled differently for prediction than for description or causal explanation. Journal of Clinical Epidemiology, 0(0). https://doi.org/10.1016/j.jclinepi.2020.03.028

Ben-David, S. (2018). Clustering—What Both Theoreticians and Practitioners are Doing Wrong. ArXiv:1805.08838 [Cs, Stat]. http://arxiv.org/abs/1805.08838

Practical highlights in my opinion:

• It's important to know about data padding in PG.
• Be conscious when modelling data tables about columns ordering, but don't be pure-school and do it in a best-effort basis.
• Gains up to 25% in wasted storage are impressive but always keep in mind the scope of the system. For me, gains are not worth it in the short-term. Whenever a system grows, it is possible to migrate data to more storage-efficient tables but mind the operative burder.

Here follows my own commands on trying the article points. I added - pg_column_size(row()) on each projection to have clear absolute sizes.

-- How does row function work?

SELECT pg_column_size(row()) AS empty,
       pg_column_size(row(0::SMALLINT)) AS byte2,
       pg_column_size(row(0::BIGINT)) AS byte8,
       pg_column_size(row(0::SMALLINT, 0::BIGINT)) AS byte16,
       pg_column_size(row(''::TEXT)) AS text0,
       pg_column_size(row('hola'::TEXT)) AS text4,
       0 AS term
;

-- My own take on that

SELECT pg_column_size(row()) AS empty,
       pg_column_size(row(uuid_generate_v4())) AS uuid_type,
       pg_column_size(row('hola mundo'::TEXT)) AS text_type,
       pg_column_size(row(uuid_generate_v4(), 'hola mundo'::TEXT)) AS uuid_text_type,
       pg_column_size(row('hola mundo'::TEXT, uuid_generate_v4())) AS text_uuid_type,
       0 AS term
;

CREATE TABLE user_order (
  is_shipped    BOOLEAN NOT NULL DEFAULT false,
  user_id       BIGINT NOT NULL,
  order_total   NUMERIC NOT NULL,
  order_dt      TIMESTAMPTZ NOT NULL,
  order_type    SMALLINT NOT NULL,
  ship_dt       TIMESTAMPTZ,
  item_ct       INT NOT NULL,
  ship_cost     NUMERIC,
  receive_dt    TIMESTAMPTZ,
  tracking_cd   TEXT,
  id            BIGSERIAL PRIMARY KEY NOT NULL
);

SELECT a.attname, t.typname, t.typalign, t.typlen
  FROM pg_class c
  JOIN pg_attribute a ON (a.attrelid = c.oid)
  JOIN pg_type t ON (t.oid = a.atttypid)
 WHERE c.relname = 'user_order'
   AND a.attnum >= 0
 ORDER BY a.attnum;

-- What is it about pg_class, pg_attribute and pg_type tables? For future investigation.

-- SELECT sum(t.typlen)
-- SELECT t.typlen
SELECT a.attname, t.typname, t.typalign, t.typlen
  FROM pg_class c
  JOIN pg_attribute a ON (a.attrelid = c.oid)
  JOIN pg_type t ON (t.oid = a.atttypid)
 WHERE c.relname = 'user_order'
   AND a.attnum >= 0
 ORDER BY a.attnum
;

-- Whoa! I need to master mocking data directly into db.

INSERT INTO user_order (
    is_shipped, user_id, order_total, order_dt, order_type,
    ship_dt, item_ct, ship_cost, receive_dt, tracking_cd
)
SELECT true, 1000, 500.00, now() - INTERVAL '7 days',
       3, now() - INTERVAL '5 days', 10, 4.99,
       now() - INTERVAL '3 days', 'X5901324123479RROIENSTBKCV4'
  FROM generate_series(1, 1000000);

-- New item to learn, pg_relation_size. 

SELECT pg_relation_size('user_order') AS size_bytes,
       pg_size_pretty(pg_relation_size('user_order')) AS size_pretty;

SELECT * FROM user_order LIMIT 1;

SELECT pg_column_size(row(0::NUMERIC)) - pg_column_size(row()) AS zero_num,
       pg_column_size(row(1::NUMERIC)) - pg_column_size(row()) AS one_num,
       pg_column_size(row(9.9::NUMERIC)) - pg_column_size(row()) AS nine_point_nine_num,
       pg_column_size(row(1::INT2)) - pg_column_size(row()) AS int2,
       pg_column_size(row(1::INT4)) - pg_column_size(row()) AS int4,
       pg_column_size(row(1::INT2, 1::NUMERIC)) - pg_column_size(row()) AS int2_one_num,
       pg_column_size(row(1::INT4, 1::NUMERIC)) - pg_column_size(row()) AS int4_one_num,
       pg_column_size(row(1::NUMERIC, 1::INT4)) - pg_column_size(row()) AS one_num_int4,
       0 AS term
;

SELECT pg_column_size(row(''::TEXT)) - pg_column_size(row()) AS empty_text,
       pg_column_size(row('a'::TEXT)) - pg_column_size(row()) AS len1_text,
       pg_column_size(row('abcd'::TEXT)) - pg_column_size(row()) AS len4_text,
       pg_column_size(row('abcde'::TEXT)) - pg_column_size(row()) AS len5_text,
       pg_column_size(row('abcdefgh'::TEXT)) - pg_column_size(row()) AS len8_text,
       pg_column_size(row('abcdefghi'::TEXT)) - pg_column_size(row()) AS len9_text,
       0 AS term
;

SELECT pg_column_size(row(''::TEXT, 1::INT4)) - pg_column_size(row()) AS empty_text_int4,
       pg_column_size(row('a'::TEXT, 1::INT4)) - pg_column_size(row()) AS len1_text_int4,
       pg_column_size(row('abcd'::TEXT, 1::INT4)) - pg_column_size(row()) AS len4_text_int4,
       pg_column_size(row('abcde'::TEXT, 1::INT4)) - pg_column_size(row()) AS len5_text_int4,
       pg_column_size(row('abcdefgh'::TEXT, 1::INT4)) - pg_column_size(row()) AS len8_text_int4,
       pg_column_size(row('abcdefghi'::TEXT, 1::INT4)) - pg_column_size(row()) AS len9_text_int4,
       0 AS term
;

SELECT pg_column_size(row(1::INT4, ''::TEXT)) - pg_column_size(row()) AS int4_empty_text,
       pg_column_size(row(1::INT4, 'a'::TEXT)) - pg_column_size(row()) AS int4_len1_text,
       pg_column_size(row(1::INT4, 'abcd'::TEXT)) - pg_column_size(row()) AS int4_len4_text,
       pg_column_size(row(1::INT4, 'abcde'::TEXT)) - pg_column_size(row()) AS int4_len5_text,
       pg_column_size(row(1::INT4, 'abcdefgh'::TEXT)) - pg_column_size(row()) AS int4_len8_text,
       pg_column_size(row(1::INT4, 'abcdefghi'::TEXT)) - pg_column_size(row()) AS int4_len9_text,
       0 AS term
;

SELECT pg_column_size(row()) - pg_column_size(row()) AS empty_row,
       pg_column_size(row(''::TEXT)) - pg_column_size(row()) AS no_text,
       pg_column_size(row('a'::TEXT)) - pg_column_size(row()) AS min_text,
       pg_column_size(row(1::INT4, 'a'::TEXT)) - pg_column_size(row()) AS two_col,
       pg_column_size(row('a'::TEXT, 1::INT4)) - pg_column_size(row()) AS round4;

SELECT pg_column_size(row()) - pg_column_size(row()) AS empty_row,
       pg_column_size(row(1::SMALLINT)) - pg_column_size(row()) AS int2,
       pg_column_size(row(1::INT)) - pg_column_size(row()) AS int4,
       pg_column_size(row(1::BIGINT)) - pg_column_size(row()) AS int8,
       pg_column_size(row(1::SMALLINT, 1::BIGINT)) - pg_column_size(row()) AS padded,
       pg_column_size(row(1::INT, 1::INT, 1::BIGINT)) - pg_column_size(row()) AS not_padded;

SELECT a.attname, t.typname, t.typalign, t.typlen
  FROM pg_class c
  JOIN pg_attribute a ON (a.attrelid = c.oid)
  JOIN pg_type t ON (t.oid = a.atttypid)
 WHERE c.relname = 'user_order'
   AND a.attnum >= 0
 ORDER BY t.typlen DESC;

DROP TABLE user_order;

CREATE TABLE user_order (
  id            BIGSERIAL PRIMARY KEY NOT NULL,
  user_id       BIGINT NOT NULL,
  order_dt      TIMESTAMPTZ NOT NULL,
  ship_dt       TIMESTAMPTZ,
  receive_dt    TIMESTAMPTZ,
  item_ct       INT NOT NULL,
  order_type    SMALLINT NOT NULL,
  is_shipped    BOOLEAN NOT NULL DEFAULT false,
  order_total   NUMERIC NOT NULL,
  ship_cost     NUMERIC,
  tracking_cd   TEXT
);

-- And, what about other varying size types as JSONB?

SELECT pg_column_size(row('{}'::JSONB)) - pg_column_size(row()) AS empty_jsonb,
       pg_column_size(row('{}'::JSONB, 0::INT4)) - pg_column_size(row()) AS empty_jsonb_int4,
       pg_column_size(row(0::INT4, '{}'::JSONB)) - pg_column_size(row()) AS int4_empty_jsonb,
       pg_column_size(row('{"a": 1}'::JSONB)) - pg_column_size(row()) AS basic_jsonb,
       pg_column_size(row('{"a": 1}'::JSONB, 0::INT4)) - pg_column_size(row()) AS basic_jsonb_int4,
       pg_column_size(row(0::INT4, '{"a": 1}'::JSONB)) - pg_column_size(row()) AS int4_basic_jsonb,
       0 AS term;

Centric web solution is the renowned best web development company.

We have a very highly experienced and expert development team who are experts in web design & development.

We provide various services like Wordpress web development, eCommerce web development, Wordpress theme design and plugin development, website maintenance & optimization.

For more our services call us on +91 98587 58541 or visit our website https://www.centricwebsolution.com/.

Our Services Are:-

• Web Design & Development
• WordPress Development
• WooCommerce Development
• Custom Web Application Development
• Website Migration Services
• Website Maintenance & Performance optimization
• Website Plugin and API Development
• Website Store Optimization
• PHP Web Development
• Enterprise Web Application Development
• Laravel Development Services
• Angularjs Development Services

感觉此文于我而言真正有价值的恐怕只有文末附录的 Dataset tables 汇总整理了。。。。。

作者提出了一种新的Alrao优化算法，让网络中每个 unit 或 feature 都各自从不同级别的随机分布中采样获得其自己的学习率。该算法没有额外计算损耗，可以更快速达到理想 lr 下的SGD性能，用来测试 DL 模型很棒！

文章声称的全局最小训练，事实上主要基于一个比较特殊的人工神经网络的结构，用了各种连接到 output 的 skip connection，还有几个额外的assumptions 作为理论保证。

这篇文章简而言之就是mini-batch sizes取得尽可能小一些可能比较好。自己瞅了一眼正在写的 paper，这不禁让我小肝微微一颤，心想：还是下次再把 batch-size 取得小一点吧。。。[挖鼻] ​​​​

最近(2018/8)在听数学与系统科学的非凸最优化进展时候，李博士就讲过：现在其实不太欣赏变 learning rate 了，反而逐步从 SGD 到 MGD 再到 GD 的方式，提高 batch-size 会有更好的优化效果！

每次看到研究梯度优化理论的 paper，都感觉到无比的神奇！厉害到爆表。。。。

这似乎是说反向传播的算法，在函数结构本身上，就还有很大的优化空间。作者在一些初等函数和常见矩阵操作基础上探索了一些操作搭配，发现效能轻易的就优于传统的反向传播算法。

不禁启发：我们为什么不能也用网络去拟合优化梯度更新函数呢？

全篇的数学理论证明：深度过参网络可以训练到0。（仅 train loss，非 test loss）+（GD，非 SGD）

烧脑！CMU、北大等合著论文真的找到了神经网络的全局最优解

又一个全篇的数学理论证明，但是没找到 conclusion 到底是啥，唯一接近的是 remark 的信息，但内容也都并不惊奇。不过倒是一个不错的材料，若作为熟悉DNN背后的数学描述的话。

It seems that the edge probe is a special case of the value probe.

@article{ben2002robust, title={Robust optimization--methodology and applications}, author={Ben-Tal, Aharon and Nemirovski, Arkadi}, journal={Mathematical Programming}, volume={92}, number={3}, pages={453--480}, year={2002}, publisher={Springer} }

Fast Threshold Clustering Algorithm

Looking for equivalent source code to apply in smart content delivery and wireless network optimisation such as Ant Mesh via @KirkDBorne's status https://twitter.com/KirkDBorne/status/479216775410626560 http://cssanalytics.wordpress.com/2013/11/26/fast-threshold-clustering-algorithm-ftca/

That's the reason why step size is an important factor in optimization algorithm. Too small step can cause the algorithm longer to converge. Too large step can cause that we change the parameters too much thus overstepped the optima.

Are there any studies that compare different pros and cons of the optimization procedures with respect to some specific NN architectures (e.g., classical LeNets)?