When a thread releases the lock, a happens-before relationship is established between that action and any subsequent acquisition of the same lock.
happens-brefore
When a thread releases the lock, a happens-before relationship is established between that action and any subsequent acquisition of the same lock.
happens-brefore
Under the Hood article: "The lean, mean virtual machine."
link broken
go https://www.infoworld.com/article/2077184/the-lean--mean--virtual-machine.html instead
two opcodes directly related to thread synchronization, the opcodes used for entering and exiting monitors.
two opcodes entering & exiting monitors
so what is monitor
?
Quota Scopes
三种 scope,主要是针对 Pod 的不同状态的
local ephemeral storage
非 memory 的 local ephemeral storage 也需要实体硬盘来支撑吧
via david, OAM
via david, mentioned by OAM guys in Ali
Built-in Environment Variables
circle-ci
docker interactive course, seems interesting
runs-on: ubuntu-latest
saved my life
via https://github.com/docker/buildx/issues/344#issuecomment-670152753
todo, what's the difference between circleci executor: machine & docker?
With or Without ‘export’
shell, With or Without ‘export’
Docker-Buildx 0.4.2
sweet, github
A context is a combination of several properties. These include
buildx 最详细的文档了。。。
docker
没太看懂和 image
的区别
via https://docs.docker.com/engine/reference/commandline/build/#specifying-external-cache-sources
This feature requires the BuildKit backend. You can either enable BuildKit or use the buildx plugin. The previous builder has limited support for reusing cache from pre-pulled images.
mode - Specifies how many layers are exported with the cache.
mode:
Metadata is always exported for the whole build.
Supported types are registry, local and inline
registry
exports build cache to a cache manifest in the registry, local
exports cache to a local directory on the client inline
writes the cache metadata into the image configuration.主要不同点
这个表格很清楚
The OCI format is a specification for container images based on the Docker Image Manifest Version 2, Schema 2 format.
OCI: Open Container Initiative
separately
?
这个才是 buiidkit 的终极文档啊
inline cache exporter only supports min cache mode.
fuck!
via https://docs.docker.com/develop/develop-images/build_enhancements/
走向质变的中美关系
奇文
You can run Buildx in different configurations that are exposed through a driver concept. Currently, Docker supports a “docker” driver that uses the BuildKit library bundled into the docker daemon binary, and a “docker-container” driver that automatically launches BuildKit inside a Docker container.
一个聊胜于无的关于 docker builder driver 的解释
circle ci, buildx
more about helm, and how to create a helm chart
nice walk through of how to use command: helm
via https://helm.sh/
to know more about the k8s ecosystem
via ruanyifeng http://www.ruanyifeng.com/blog/2020/09/weekly-issue-127.html
漫画家斯科特·亚当斯(Scott Adams)曾经提过一种建立个人护城河的方法,就是找到自己最擅长的2个~3个事物的交集。比如,他既不是最好的漫画家,也不是最好的作家,也不是最好的企业家,但他可以是最好的商业类漫画短文作者,这就是他的护城河。
Java 内存模型是个很复杂的规范,可以从不同的视角来解读,站在我们这些程序员的视角,本质上可以理解为,Java 内存模型规范了 JVM 如何提供按需禁用缓存和编译优化的方法。具体来说,这些方法包括 volatile、synchronized 和 final 三个关键字,以及六项 Happens-Before 规则,这也正是本期的重点内容。
这个解读有点儿意思
感觉这个专栏比较水,写的不深入,太过范范
github action, build img with cache
nice intro of docker multi-stage build
If an environment variable is only needed during build, and not in the final image, consider setting a value for a single command instead:
ENV
会在 container 中也生效,如果没有这个需求,而只是希望在 build 阶段用到变量,应该使用 ARG
The ARG instruction defines a variable that users can pass at build-time to the builder with the docker build command using the --build-arg <varname>=<value> flag
build image 的入参
founder of a small SaaS company: https://www.bannerbear.com/
nice saas tool for generating pics for/from sns
The parameters are organized by order of importance, ranked from high to low.
really nice
replicated three ways
3 ways 啥意思?
“Ongoing,” “Prepare commit,” and “Completed.”
Transaction states:
watch the Kafka summit talk where transactions were introduced.
a more in-depth dive into Envoy from the Author
via Envoy intro video: https://www.youtube.com/watch?v=D0cuv1AEftE
The PUT and POST verbs on objects MUST ignore the "status" values, to avoid accidentally overwriting the status in read-modify-write scenarios. A /status subresource MUST be provided to enable system components to update statuses of resources they manage.
status
需要单独的接口更新
<del>?</del>
When a TCP packet carrying some of those bytes is lost on the network path, it creates a gap in the stream and TCP needs to fill it by resending the affected packet when the loss is detected. While doing so, none of the successfully delivered bytes that follow the lost ones can be delivered to the application, even if they were not themselves lost and belong to a completely independent HTTP request. So they end up getting unnecessarily delayed as TCP cannot know whether the application would be able to process them without the missing bits. This problem is known as “head-of-line blocking”.
head-of-line blocking
HTTP3
HTTP2
QUIC 功能
HTTP/2 特性
via tianfang
nice article explaining HTTP/1.1 keep-alive
a runnable demo worth thousand words
We can actually check this difference using telnet.
really cool
➜ telnet -4 taobao.com 80
Trying 140.205.94.189...
Connected to taobao.com.
Escape character is '^]'.
HEAD / HTTP/1.0
host: taobao.com
HTTP/1.1 302 Found
Server: Tengine
Date: Fri, 31 Jul 2020 03:21:12 GMT
Content-Type: text/html
Content-Length: 258
Connection: close <<<<<<<<<<<<<<<<<<<<<<<
Location: http://www.taobao.com/
➜ telnet -4 taobao.com 80
Trying 140.205.220.96...
Connected to taobao.com.
Escape character is '^]'.
HEAD / HTTP/1.1
host: taobao.com
HTTP/1.1 302 Found
Server: Tengine
Date: Fri, 31 Jul 2020 03:20:53 GMT
Content-Type: text/html
Content-Length: 258
Connection: keep-alive <<<<<<<<<<<<<<<<<<<<<<<
Location: http://www.taobao.com/
free and open source RSS reader
mark
keep in mind that no built-in controllers are running in the test context
no build-in controllers are running
The object is still visible via the REST API
still visible for objects in deletion
another service mesh, refed by david
recommended by david
When a Certificate is created, a corresponding CertificateRequest resource is created by cert-manager containing the encoded x509 certificate request, Issuer reference, and other options based upon the specification of the Certificate resource.
这里 Certificate
更像是一种声明或者说意图,需要 Issuer 的进一步处理,才能得到我们想象中的 certificate key pair
CertificateRequest is created by cert-manager containing:
generated labels
{
app_name="ddex-api"
instance="100.96.17.155:9091"
job="ddex-metrics"
key="v3/POST-buildOrder"
le="10"
namespace="production"
node_name="nodes-c507"
pod_name="ddex-api-5cb9b89f58-7fnbc"
ready="true"
} 0
an endpoint you can scrape is called an instance, usually corresponding to a single process. A collection of instances with the same purpose, a process replicated for scalability or reliability for example, is called a job.
怪异的名字
比如 api 服务,在k8s 中以 deployment 部署,scale 为 5。
则 api-1... api-5 是一个个的 instance,而这五个pod,是一个job
这篇好啊,特别是这张对应代码的图,对Heap & Stack 的作用展示的很清晰
via limin
also heavily mentioned in book: k8s operator
alternative for kubebuilder? this one get more starts
via tianfang
k8s dashboard
As circuit breaking applies to “real” mesh destinations in a load balancing pool, you configure circuit breaker thresholds in destination rules, with the settings applying to each individual host in the service.
还是没懂这个怎么就放在 DestinationRules
上了,前面都是配置在 VirtualService
上
另外这个 circuit breaker 的规则是在每一个 host 上生效,如果一个服务有多个 pods,会每个 pod 去连一遍么,还是单个失败了,就直接短路
Istio failure recovery features are completely transparent to the application
? 怎么个透明法?
You don’t need to add a service entry for every external service that you want your mesh services to use. By default, Istio configures the Envoy proxies to passthrough requests to unknown services.
未知的服务 Envoy 会放行,所以并不需要每个外部服务都做配置?
mentioned in 再有人问你分布式事务,把这篇扔给他
mentioned in 美团外卖订单中心的演进
https://tech.meituan.com/2016/09/09/mt-waimai-order-evolution.html
MARK
Exercise: Loop over the string using the %q format on each byte. What does the output tell you?]
?
'½''²''=''¼'' ''â''\u008c''\u0098'
Some resources in the v1 API contain fields called phase, and associated message, reason, and other status fields. The pattern of using phase is deprecated
phase
is deprecated, use conditions
instead...
Conditions represent the latest available observations of an object's state.
status.conditions
PUT expects the whole object to be specified. Therefore, if a field is omitted it is assumed that the client wants to clear that field's value. The PUT verb does not accept partial updates.
PUT 是覆盖式的
For example a client may acquire the lock, get blocked in some operation for longer than the lock validity time (the time at which the key will expire), and later remove the lock, that was already acquired by some other client.
即使加上后面的乐观锁检查,
这种设计下,还是有corner case:
当 原先持有锁的 A因为某些原因暂停了一段时间,导致自己失去了锁,等它醒来之后锁已经被 B 获取,而 A 还以为自己有锁,这时会出现一个短暂的时间段,A 和 B 都认为自己有锁,BOOM!
反驳的文章也提到了这一点,解决的办法是使用乐观锁,例如数据库里记录增加 version 字段
不过这时候,例如数据库有乐观锁的支持,那也不需要靠redis 来提供分布式锁的支持了。
split brain conditions
?
via limin 推荐的面试准备材料
via Distributed Lock with Redis
写的还行,Redis v.s. ZK
learning notes on grokking-system-design
https://www.educative.io/courses/grokking-the-system-design-interview
feedbacks on: grokking the system design interview
MARK
MARK
a B-tree is a self-balancing tree data structure that maintains sorted data and allows searches, sequential access, insertions, and deletions in logarithmic time. The B-tree generalizes the binary search tree, allowing for nodes with more than two children.
key word:
InnoDB 中用于存储数据的文件总共有两个部分,一是系统表空间文件
二呢?。。。
默认情况下,表空间中的页大小都为 16KB
一个 page 挺小的啊,16k,只能存几行记录
via PingCAP
base for k8s.io/code-generator ?
mentioned in k8s/code-generator official doc:
学生, 毕业帽头像生成
To control the cascading deletion policy, set the propagationPolicy field on the deleteOptions argument when deleting an Object. Possible values include “Orphan”, “Foreground”, or “Background”.
删除 owner 时,可以指定级联删除的策略,包括:
x509.Certificate in Go
Indexer: An indexer provides indexing functionality over objects.
还是不太懂这个 Indexer 是干啥的,哪儿能用到
以 192.77
Sai / ETH 的价格
将 3456.79
个 Sai 换成了 17.9
个 ETH
0xb4 的地址剩余 10Sai ,看看 Sai 关停后处理流程是啥
gitops
via tianfang
深分页
文章不错不过对面试帮助不大,因为面试要涉及的几个概念,文章都没太讲到:
While the Marking work is happening on P1, application work can continue concurrently on P2, P3 and P4.
GC runs with user app
The only way to do that is for the collector to watch and wait for each goroutine to make a function call.
make a function call?
Nice animation on GC
author of calibre https://calibre-ebook.com/
via waylybaye
expose the service using an Ingress resource that will be handled by istio-ingressgateway.
感觉这个 Ingress 没啥作用,和 Istio 的 Gateway 有重复
Istio as a secure Kubernetes Ingress controller
Istio now is the controller of our k8s Ingress
Reference for Kapp tutorial
using the given private key
CA 是自己又一个私钥,用来签其他的证书,而 selfsigned 就是自己提供私钥自己签。
SelfSigned
没懂这个是用来干啥的
The relevant extension for Extended Validation (EV) is Certificate Policies. Certificate sellers will refuse to issue wildcard certificates with EV, because cabforum.org, the regulatory body governing the issuance of EV SSL Certificates decided this is a big no no. EV certificates can, however, have as much SubjectAltName as you wish.
? 在说啥。。。
C=PT, ST=Lisboa, L=Lisboa, O=Example Org, CN=*.example.org
entity
C=PT, ST=Lisboa, L=Lisboa, O=Sz CA, OU=SZ CA
Issuer
Take note of the index keys used for each field as these are required in order for cert-manager to find the certificate and key.
这儿意思是说 secret 的 fields 是固定的,必须是:
?
acme-crt-secret
refed issuer will try to obtain a certificate key pair for domains, and stored in this secret: acme-crt-sercret
, with keys of:
Unlike the previous sections, the Istio default ingress gateway will not work out of the box because it is only preconfigured to support one secure host
只支持一个 TLS 域名的时候不需要修改 istio-ingressgateway 是因为它做了预配置以支持一个 TLS Host
Redeploy istio-ingressgateway with the new certificate
Gateway 支持多个证书时,File Mount 的方法需要重新部署 istio-ingressgateway ,这是它的最大缺点
重新部署是因为需要把新证书 mount 到 istio-ingressgateway 的文件系统
目前还看不太懂。。。
不过看评论文章似乎不错
Design goals
design goals 里,这两点明白意思
剩下的:
不懂在说啥。。。
Traffic in Istio is categorized as data plane traffic and control plane traffic.
traffic in Istio:
Great article on Demystifying Istio's Sidecar istio-proxy
We did briefly mention that it is done by setting up the iptable rules within the pod namespace, which in turn is done by the istio-init container.
粗略的说,通过修改 pod 的 iptable
rules 来使 sidecar proxy 完成进出流量的截取,而 iptables
的修改是通过 init-container 来完成的。
那更进一步的细节呢?见下文
An admission controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized.
admission controller 会拦截发往 k8s API Server 的请求,拦截的时机在
two types of admission webhooks, validating admission Webhook and mutating admission webhook
2 types of admission webhooks:
The static_resources block contains definitions for clusters and listeners that aren’t dynamically managed. A cluster is a named group of hosts/ports, over which Envoy will load balance traffic, and listeners are named network locations that clients can connect to.
static_resources:
listerners: [] # A cluster is a named group of hosts/ports, over which Envoy will load balance traffic
clusters:[] # listeners are named network locations that clients can connect to
In other words, the Gateway resource must reside in the same namespace as the gateway workload instance.
?
Gateway 必须和 Istio 的 proxy 配置在同一个 namespace 中?也就是 Gateway 必须配置在 istio-system 中?
貌似并不是这样
One or more labels that indicate a specific set of pods/VMs on which this gateway configuration should be applied.
Istio 的 Gateway 和 k8s 原生的 Ingress 一样,其实都是一种对真正的 Proxy 的配置
While Istio will configure the proxy to listen on these ports, it is the responsibility of the user to ensure that external traffic to these ports are allowed into the mesh.
?
The gateway will be applied to the proxy running on a pod with labels app: my-gateway-controller.
Gateway
spec.selector
Share u with the json.Unmarshal function.
&u, address of pointer u?
Here is an example of a stack that is replaced several times because of growth.
没太看懂
Escape analysis is the process that the compiler uses to determine the placement of values that are created by your program.
Escape Analysis: 编译器用来决定内存的分配位置,stack or heap
Pointer variables are not special because they are variables like any other variable. They have a memory allocation and they hold a value.
指针变量并不特殊,也需要对应的分配内存,有对应的值
This is what the stack looks like after the function call to increment:
这个图还蛮好的,帮助理解 stack、值、值的指针
pointer 让跨 栈帧 的内存共享成为可能
传递的是 Pointer, &count
Variables serve the purpose of assigning a name to a specific memory location for better code readability
变量是内存地址的一个代号,方便代码的阅读
基本了解了 逃逸分析 是干什么的:编译器通过确定变量是分配在 heap 还是 stack 上。
不过行文感觉比较随意
Since a thread cannot both be executing code and be blocked on a syscall
why not
意思是说一个线程在因为 系统调用被block之后,不能去执行其他代码?
图还挺好看的,但是没讲啥是内存屏障啊
李笑来旗下做的在线音频聊天室
https://weibo.cn/comment/IFCnq887F?uid=1088413295&rl=0&gid=10001#cmtfrm
在不获取手机麦克风权限的情况下,通过陀螺仪传感器数据的神经网络算法,就可以从声波振动中提取出讲话者的语音。
采用DH算法后,Premaster secret不需要传递,双方只要交换各自的参数,就可以算出这个随机数。
不明觉厉
via Istio
Mapping a single virtual service to multiple “real” services is particularly useful in facilitating turning a monolithic application into a composite service built out of distinct microservices without requiring the consumers of the service to adapt to the transition.
把一个 胖服务逐渐分拆?
By contrast, container orchestration platforms like Kubernetes only support traffic distribution based on instance scaling, which quickly becomes complex.
instance scaling 指的什么?pod 的伸缩?
it’s far from all that Istio can do
first meet in Istio doc
https://istio.io/docs/concepts/what-is-istio/#platform-support
also, this hashicorp seems to be an interesting company
while replace directives in modules other than the main module are ignored when building the main module.
这个怎么理解?
You'll often hear experienced Go programmers talk about the "slice header" because that really is what's stored in a slice variable.
sliceHeader
Metric types
4 metric types:
Samples
一个 sample 就是一个数据点,包括:
一条 time serial 由多个 samples 组成
via this simple & clear article on k8s AdmissionWebhook
https://github.com/kelseyhightower/denyenv-validating-admission-webhook
simple & clear article on AdmissionWebhook
Authentication strategies
strategies:
Requeue when we either see a running job (done automatically) or it’s time for the next scheduled run.
没太懂这个 Requeue 是怎么工作的
Reconcile
when this will be called?
重点
don’t need to try again
don't need to reconcile this cronjob object again?
A reconciler takes the name of an object, and returns whether or not we need to try again
try again what?
except to mark that we want a status subresource, so that we behave like built-in kubernetes types
what subresource?
That little +kubebuilder:object:root comment is called a marker. We’ll see more of them in a bit, but know that they act as extra metadata, telling controller-tools (our code and YAML generator) extra information. This particular one tells the object generator that this type represents a Kind. Then, the object generator generates an implementation of the runtime.Object interface for us, which is the standard interface that all types representing Kinds must implement.
about the mysterious marker:
+kubebuilder:object:root
A resource is simply a use of a Kind in the API.
多数时候,Kind 和 resource 是一对一的,例如:
pods
resource 和 Pod
Kind
但有时候,一个 Kind 对应多个 resources,例如:
Scale
Kind 对应多个 scale subresources:deployments/scale
and replicasets/scale
不知所云,主要是不知道 webhook 这儿是干啥的
The aggregation layer
where is aggregation layer?
extension points
so what is Extension Points...
Kubernetes is designed to be automated by writing client programs
? 怎么就 automated 了?
base search domain
for None-FQDN
Virtual IPs and service proxies
不懂下面几小节和 Virtual IPs and service proxies 的关系
可能是说 kueb-proxy 使用这几种模式来支持 Service
ServiceTypes
4:
A cluster-aware DNS server, such as CoreDNS, watches the Kubernetes API for new Services and creates a set of DNS records for each one.
一些 cluster-ware 的 DNS 服务,例如 CoreDNS,会通过 k8s API 来监视 Service 的变动,并对应的生成 DNS 记录
Kubernetes supports 2 primary modes of finding a Service - environment variables and DNS.
2 primary modes of finding a Service:
User space proxy mode
... 都是高科技。。
介个实例教程还挺好,总算大致了解 Stateful Set
的 特点/存在理由 了
Pod initialization
我自己看见的几个值得注意的点:
2: 检查如果是 slave pod 第一次启动,则从前一个 pod 同步数据,已加速之后和 master 的同步
MVCC
Multi-Version Concurrency Control: MVCC
在可重复读中,该sql第一次读取到数据后,就将这些数据加锁,其它事务无法修改这些数据,就可以实现可重复读了
和上面例子有冲突
B 确实修改了 A 读出来的数据,并没有被锁挡住
why?
事务B也就一直拿不到该行锁,wait直到超时。
如果这儿 B 事务是读呢?
是怎么保证 B 不会读到尚未提交的修改的?
Because slices are multiword structures, not pointers, the slicing operation does not need to allocate memory, not even for the slice header, which can usually be kept on the stack
?
stack 不算 memory?
按照我个人的理解,在事务处理的ACID属性中,一致性是最基本的属性,其它的三个属性都为了保证一致性而存在的。
原子性和隔离是为了保证一致性
隔离级别的实现
ACID
例如 PostgreSQL 会为每一条记录保留两个字段;读时间戳中包括了所有访问该记录的事务中的最大时间戳,而记录行的写时间戳中保存了将记录改到当前值的事务的时间戳。
看起来 PG 和 MySQL 这儿是个很大的不同点呀
Next-Key 这一块儿没讲清楚
InnoDB 不仅会在范围 (21, 30] 上加 Next-Key 锁,还会在这条记录后面的范围 (30, 40] 加间隙锁
所以这儿的 (21, 30] 的 Next-Key 锁和 (30,40] 的间隙锁,有什么区别?
意向锁其实不会阻塞全表扫描之外的任何请求,它们的主要目的是为了表示是否有人请求锁定表中的某一行数据
?
不可重复读的原因就是,在 READ COMMITED 的隔离级别下,存储引擎不会在查询记录时添加行锁,锁定 id = 3 这条记录。
🔒,行锁
第 4、5 步就是在事务提交时执行的
如果事务提交后,
重做日志由两部分组成,一是内存中的重做日志缓冲区,因为重做日志缓冲区在内存中,所以它是易失的,另一个就是在磁盘上的重做日志文件,它是持久的。
redo log:
重做日志(redo log)
前面保证事务的是 undo log
这儿保证持久性的是 redo log
一旦事务被提交,那么数据一定会被写入到数据库中并持久存储起来
反过来也是在说 事务提交 和 数据被写入数据库(或者说写到磁盘) 不是同一件事