It's interesting that from the human perspective we are just spelling out the word "Happy Birthday" as a whole, but never come to realize or pay attention to the fact that the string is actually a combination of the different letters taht make up the word.

This reminds me most of measuring value of life in systems such as trolley problems or AI car decision making. Is a doctor more worthy of being saved than a musician? Or a depressed person? Or a felon? Where do you draw the line? If you draw a line, how many "felon lives" equals one doctor life? Utilitarianism to me isn't a morality system itself but a coping mechanism to allow humans to rationalize tough decisions. But when humans put the same logic in computers, it's not a coping strategy for a computer's feelings, but just a flawed series of priorities.

Welcome back and in this brief demo lesson I want to give you some experience of working with both EC2 instance connect as well as connecting with a local SSH client.

Now these are both methods which are used for connecting to EC2 instances both with public IP version 4 addressing and IP version 6 addressing.

Now to get started we're going to need some infrastructure so make sure that you're logged in as the IAM admin user into the general AWS account which is the management account of the organization and as always you'll need the northern Virginia region selected.

Now in this demonstration you are going to be connecting to an EC2 instance using both instance connect and a local SSH client and to use a local SSH client you need a key pair.

So to create that let's move across to the EC2 console, scroll down on the left and select key pairs.

Now you might already have key pairs created from earlier in the course.

If you have one created which is called A4L which stands for Animals for Life then that's fine.

If you don't we're going to go ahead and create that one.

So click on create key pair and then under name we're going to use A4L.

Now if you're using Windows 10 or Mac OS or Linux then you can select the PEM file format.

If you're using Windows 8 or prior then you might need to use the putty application and to do that you need to select PPK.

But for this demonstration I'm going to assume that you're using the PEM format.

So again this is valid on Linux, Mac OS or any recent versions of Microsoft Windows.

So select PEM and then click on create key pair and when you do it's going to present you with a download.

It's going to want you to save this key pair to your local machine so go ahead and do that.

Once you've done that from the AWS console attached to this lesson is a one-click deployment link.

So I want you to go ahead and click that link.

That's going to move you to a quick create stack screen.

Everything should be pre-populated.

The stack name should be EC2 instance connect versus SSH.

The key name box should already be pre-populated with A4L which is a key that you just created or one which you already had.

Just move down to the very bottom, check the capabilities box and then click on create stack.

Now you're going to need this to be in a create complete state before you continue with the demo lesson.

So pause the video, wait for your stack to change to create complete and then you're good to continue.

Okay so this stacks now in a create complete status and we're good to continue.

Now if we click on the resources tab you'll see that this has created the standard animals for life VPC and then it's also created a public EC2 instance.

So this is an EC2 instance with a public IP version 4 address that we can use to connect to.

So that's what we're going to do.

So click on services and then select EC2 to move to the EC2 console.

Once you're there click on instances running and you should have a single EC2 instance A4L-publicEC2.

Now the two different ways which I want to demonstrate connecting to this instance in this demo lesson are using a local SSH client and key based authentication and then using the EC2 instance connect method.

And I want to show you how those differ and give you a few hints and tips which might come in useful for production usage and for the exams.

So if we just go ahead and select this instance and then click on the security tab you'll see that we have this single security group which is associated to this instance.

Now make sure the inbound rules is expanded and just have a look at what network traffic is allowed by this security group.

So the first line allows port 80 TCP which is HTTP and it allows that to connect to the instance from any source IP address specifically IP version 4.

We can tell it's IP version 4 because it's 0.0.0.0/0 which represents any IP version 4 address.

Next we allow port 22 using TCP and again using the IP version 4 any IP match and this is the entry which allows SSH to connect into this instance using IP version 4.

And then lastly we have a corresponding line which allows SSH using IP version 6.

So we're allowing any IP address to connect using SSH to this EC2 instance.

And so connecting to it using SSH is relatively simple.

We can right click on this instance and select connect and then choose SSH client and AWS provides us with all of the relevant information.

Now note how under step number three we have this line which is chmod space 400 space a4l.pm.

I want to demonstrate what happens if we attempt to connect without changing the permissions on this key file.

So to do that right at the bottom is an example command to connect to this instance.

So just copy that into your clipboard.

Then I want you to move to your command prompt or terminal.

In my case I'm running macOS so I'm using a terminal application.

Then you'll need to move to the folder where you have the PEM file stored or where you just downloaded it in one of the previous steps.

I'm going to paste in that command which I just copied onto my clipboard.

This is going to use the a4l.pm file as the identity information and then it's going to connect to the instance using the EC2-user local Linux user.

And this is the host name that it's going to connect to.

So this is my EC2 instance.

Now I'm going to press enter and attempt that connection.

First it will ask me to verify the authenticity of this server.

So this is an added security method.

This is getting the fingerprint of this EC2 instance.

And it means that if we independently have a copy of this fingerprint, say from the administrator of the server that we're connecting to, then we can verify that we're connecting to that same server.

Because it's possible that somebody could exploit DNS and replace a legitimate DNS name with one which points at a non-legitimate server.

So that's important.

You can't always rely on a DNS name.

DNS names can be adjusted to point at different IP addresses.

So this fingerprint is a method that you can use to verify that you're actually connecting to the machine or the instance which you think you are.

Now in this case, because we've just created this EC2 instance, we can be relatively certain that it is valid.

So we're just going to go ahead and type yes and press enter.

And then it will try to connect to this instance.

Now immediately in my case, I got an error.

And this error is going to be similar if you're using macOS or Linux.

If you're using Windows, then there is a chance that you will get this error or won't.

And if you do get it, it might look slightly different.

But look for the keyword of permissions.

If you see that you have a permissions problem with your key, then that's the same error as I'm showing on my screen now.

Basically what this means is that the SSH client likes it when the permissions on these keys are restricted, restricted to only the user that they belong to.

Now in my case, the permissions on this file are 644.

And this represents my user, my group, and then everybody.

So this means this key is accessible to other users on my local system.

And that's far too open to be safe when using local SSH.

Now in Windows, you might have a similar situation where other users of your local machine have read permissions on this file.

What this error is telling us to do is to correct those permissions.

So if we go back to the AWS console, this is the command that we need to run to correct those permissions.

So copy that into your clipboard, move back to your terminal, paste that in, and press enter.

And that will correct those permissions.

Now under Windows, the process is that you need to edit the permissions of that file.

So right click properties and then edit the security.

And you need to remove any user access to that file other than your local user.

And that's the same process that we've just done here, only in Windows it's GUI based.

And under Mac OS or Linux, you use CHmod.

So now that we've adjusted those permissions, if I use the up arrow to go back to the previous command and press enter, I'm able to connect to the CC2 instance.

And that's using the SSH client.

To use the SSH client, you need to have network connectivity to the CC2 instance.

And you need to have a valid SSH key pair.

So you need the key stored on your local machine.

Now this can present scalability issues because if you need to have a large team having access to this instance, then everybody in that team need a copy of this key.

And so that does present admin problems if you're doing it at scale.

Now in addition to this, because you're connecting using an SSH client from your local machine, you need to make sure that the security group of this instance allows connections from your local machines.

So in this case, it allows connections from any source IP address into this instance.

And so that's valid for my IP address.

You need to make sure that the security group on whichever instance you're attempting to connect to allows your IP address as a minimum.

Now another method that you can use to connect to EC2 is EC2 instance connect.

Now to use that, we right click, we select connect, and we have a number of options at the top.

One of these is the SSH client that we've just used.

Another one is EC2 instance connect.

So if we select this option, we're able to connect to this instance.

It shows us the instance ID, it shows us the public IP address, and it shows us the user to connect into this instance with.

Now AWS attempt to automatically determine the correct user to use.

So when you launch an instance using one of the default AMIs, then it tends to pick correctly.

However, if you generate your own custom AMI, it often doesn't guess correctly.

And so you need to make sure that you're using the correct username when connecting using this method.

But once you've got the correct username, you can just go ahead and click on connect, and then it will open a connection to that instance using your web browser.

It'll take a few moments to connect, but once it has connected, you'll be placed at the terminal of this EC2 instance in exactly the same way as you were when using your local SSH.

Now one difference you might have noticed is at no point where you prompted to provide a key.

When you're using EC2 instance connect, you're using AWS permissions to connect into this instance.

So because we're logged in using an admin user, we have those permissions, but you do need relevant permissions added to the identity of whoever is using instance connect to be able to connect into the instance.

So this is managed using identity policies on the user, the group or the role, which is attempting to access this instance.

Now one important element of this, which I want to demonstrate, if we go back to instances and we select the instance, click on security, and then click on the security group, which is associated with this instance.

Scroll down, click on edit inbound rules, and then I want you to locate the inbound rule for IP version 4 SSH, SSH TCP 22, and then it's using this catchall, so 0.0.0.0/0, which represents any IP version 4 address.

So go ahead and click on the cross to remove that, and then on that same line in the source area, click on this drop down and change it to my IP.

So this is my IP address, yours will be different, but then we're going to go ahead and save that rule.

Now just close down the tab that you've got connected to instance connect, move back to the terminal, and type exit to disconnect from that instance, and then just rerun the previous command.

So connect back to that instance using your local SSH client.

You'll find that it does reconnect because logically enough, this connection is coming from your local IP address, and you've changed the security group to allow connections from that address, so it makes sense that this connection still works.

Moving back to the console though, let's go to the EC2 dashboard, go to running instances, right click on this instance, go to connect, select EC2 instance connect, and then click on connect and just observe what happens.

Now you might have spent a few minutes waiting for this to connect, and you'll note that it doesn't connect.

Now this might seem strange at this point because you're connecting from a web browser, which is running on your local machine.

So it makes sense that if you can connect from your local SSH client, which is also running on your local machine, you should be able to connect using EC2 instance connect.

Now this might seem logical, but the crucial thing about EC2 instance connect is that it's not actually originating connections from your local machine.

What's happening is that you're making a connection through to AWS, and then once your connection arrives at AWS, the EC2 instance connect service is then connecting to the EC2 instance.

Now what you've just done is you've edited the security group of this instance to only allow your local IP address to connect, and this means that the EC2 instance connect service can no longer connect to this instance.

So what you need in order to allow the EC2 instance connect service to work is you either need to allow every source IP address, so 0.0.0.0.0/0, but of course that's bad practice for production usage.

It's much more secure if you go to this URL, and I'll make sure that I include this attached to this lesson.

This is a list of all of the different IP ranges which AWS use for their services.

Now because I have this open in Firefox, it might look a little bit different.

If I just go to raw data, that might look the same as your browser.

If you're using Firefox, you have the ability to open this as a JSON document.

Both of them show the same data, but when it's JSON, you have the ability to collapse these individual components.

But the main point about this document is that this contains a list of all of the different IP addresses which are used in each different region for each different service.

So if we wanted to allow EC2 instance connect for a particular region, then we might search for instance, locate any of these items which have EC2 instance connect as the service, and then just move through them looking for the one which matches the region that we're using.

Now in my case, I'm using US East One, so I'd scroll through all of these IP address ranges looking for US East One.

There we go, I've located it.

It's using this IP address range.

So I might copy this into my clipboard, move back to the EC2 console, select the instance, click on security, select the security group of this instance, scroll down, edit the inbound rules, remove the entry for my IP address, paste in the entry for the EC2 instance connect service, and then save that rule.

And now what you'll find if you move back to your terminal and try to interact with this instance, you might be able to initially because the connection is still established, but if you exit and then attempt to reconnect, this time you'll see that you won't be able to connect because now your local IP address is no longer allowed to connect to this instance.

However, if you move back to the AWS console, go to the dashboard and then instance is running, right click on the instance and put connect, select instance connect and then click on connect.

Now you'll be allowed to connect using EC2 instance connect.

And the reason for that just to reiterate is that you've just edited the security group of this EC2 instance and you've allowed the IP address range of the EC2 instance connect service.

So now you can connect to this instance and you could do so at scale using AWS permissions.

So I just wanted to demonstrate how both of those connection methods work, both instance connect and using a local SSH client.

That's everything I wanted to cover.

So just go ahead and move back to the CloudFormation console, select this stack that you created using the one click deployment, click on delete and then confirm that process.

And that will clear up all of the infrastructure that you've used in this demo lesson.

At this point though, that's everything I wanted to cover.

So go ahead, complete this video and when you're ready, I'll look forward to you joining me in the next.

Author Response:

Reviewer #1 (Public review):

In this study, Deshmukh et al. provide an elegant illustration of Haldane's sieve, the population genetics concept stating that novel advantageous alleles are more likely to fix if dominant because dominant alleles are more readily exposed to selection. To achieve this, the authors rely on a uniquely suited study system, the female-polymorphic butterfly Papilio polytes.

Deshmukh et al. first reconstruct the chronology of allele evolution in the P. polytes species group, clearly establishing the non-mimetic cyrus allele as ancestral, followed by the origin of the mimetic allele polytes/theseus, via a previously characterized inversion of the dsx locus, and most recently, the origin of the romulus allele in the P. polytes lineage, after its split from P. javanus. The authors then examine the two crucial predictions of Haldane's sieve, using the three alleles of P. polytes (cyrus, polytes, and romulus). First, they report with compelling evidence that these alleles are sequentially dominant, or put in other words, novel adaptive alleles either are or quickly become dominant upon their origin. Second, the authors find a robust signature of positive selection at the dsx locus, across all five species that share the polytes allele.

In addition to exquisitely exemplifying Haldane's sieve, this study characterizes the genetic differences (or lack thereof) between mimetic alleles at the dsx locus. Remarkably, the polytes and romulus alleles are profoundly differentiated, despite their short divergence time (< 0.5 my), whereas the polytes and theseus alleles are indistinguishable across both coding and intronic sequences of dsx. Finally, the study reports incidental evidence of exon swaps between the polytes and romulus alleles. These exon swaps caused intermediate colour patterns and suggest that (rare) recombination might be a mechanism by which novel morphs evolve.

This study advances our understanding of the evolution of the mimicry polymorphism in Papilio butterflies. This is an important contribution to a system already at the forefront of research on the genetic and developmental basis of sex-specific phenotypic morphs, which are common in insects. More generally, the findings of this study have important implications for how we think about the molecular dynamics of adaptation. In particular, I found that finding extensive genetic divergence between the polytes and romulus alleles is striking, and it challenges the way I used to think about the evolution of this and other otherwise conserved developmental genes. I think that this study is also a great resource for teaching evolution. By linking classic population genetic theory to modern genomic methods, while using visually appealing traits (colour patterns), this study provides a simple yet compelling example to bring to a classroom.

In general, I think that the conclusions of the study, in terms of the evolutionary history of the locus, the dominance relationships between P. polytes alleles, and the inference of a selective sweep in spite of contemporary balancing selection, are strongly supported; the data set is impressive and the analyses are all rigorous. I nonetheless think that there are a few ways in which the current presentation of these data could lead to confusion, and should be clarified and potentially also expanded.

We thank the reviewer for the kind and encouraging assessment of our work.

(1) The study is presented as addressing a paradox related to the evolution of phenotypic novelty in "highly constrained genetic architectures". If I understand correctly, these constraints are assumed to arise because the dsx inversion acts as a barrier to recombination. I agree that recombination in the mimicry locus is reduced and that recombination can be a source of phenotypic novelty. However, I'm not convinced that the presence of a structural variant necessarily constrains the potential evolution of novel discrete phenotypes. Instead, I'm having a hard time coming up with examples of discrete phenotypic polymorphisms that do not involve structural variants. If there is a paradox here, I think it should be more clearly justified, including an explanation of what a constrained genetic architecture means. I also think that the Discussion would be the place to return to this supposed paradox, and tell us exactly how the observations of exon swaps and the genetic characterization of the different mimicry alleles help resolve it.

The paradox that we refer to here is essentially the contrast of evolving new adaptive traits which are genetically regulated, while maintaining the existing adaptive trait(s) at its fitness peak. While one of the mechanisms to achieve this could be differential structural rearrangement at the chromosomal level, it could arise due to alternative alleles or splice variants of a key gene (caste determination in Cardiocondyla ants), and differential regulation of expression (the spatial regulation of melanization in Nymphalid butterflies by ivory lncRNA). In each of these cases, a new mutation would have to give rise to a new phenotype without diluting the existing adaptive traits when it arises. We focused on structural variants, because that was the case in our study system, however, the point we were making referred to evolution of novel traits in general. We will add a section in the revised discussion to address this.

(2) While Haldane's sieve is clearly demonstrated in the P. polytes lineage (with cyrus, polytes, and romulus alleles), there is another allele trio (cyrus, polytes, and theseus) for which Haldane's sieve could also be expected. However, the chronological order in which polytes and theseus evolved remains unresolved, precluding a similar investigation of sequential dominance. Likewise, the locus that differentiates polytes from theseus is unknown, so it's not currently feasible to identify a signature of positive selection shared by P. javanus and P. alphenor at this locus. I, therefore, think that it is premature to conclude that the evolution of these mimicry polymorphisms generally follows Haldane's sieve; of two allele trios, only one currently shows the expected pattern.

We agree with the reviewer that the genetic basis of f. theseus requires further investigation. f. theseus occupies the same level on the dominance hierarchy of dsx alleles as f. polytes (Clarke and Sheppard, 1972) and the allelic variant of dsx present in both these female forms is identical, so there exists just one trio of alleles of dsx. Based on this evidence, we cannot comment on the origin of forms theseus and polytes. They could have arisen at the same time or sequentially. Since our paper is largely focused on the sequential evolution of dsx alleles through Haldane’s sieve, we have included f. theseus in our conclusions. We think that it fits into the framework of Haldane’s sieve due to its genetic dominance over the non-mimetic female form. However, this aspect needs to be explored further in a more specific study focusing on the characterization, origin, and developmental genetics of f. theseus in the future.

Reviewer #2 (Public review):

Summary:

Deshmukh and colleagues studied the evolution of mimetic morphs in the Papilio polytes species group. They investigate the timing of origin of haplotypes associated with different morphs, their dominance relationships, associations with different isoform expressions, and evidence for selection and recombination in the sequence data. P. polytes is a textbook example of a Batesian mimic, and this study provides important nuanced insights into its evolution, and will therefore be relevant to many evolutionary biologists. I find the results regarding dominance and the sequence of events generally convincing, but I have some concerns about the motivation and interpretation of some other analyses, particularly the tests for selection.

We thank the reviewer for these insightful remarks.

Strengths:

This study uses widespread sampling, large sample sizes from crossing experiments, and a wide range of data sources.

We appreciate this point. This strength has indeed helped us illuminate the evolutionary dynamics of this classic example of balanced polymorphism.

Weaknesses:

(1) Purpose and premise of selective sweep analysis

A major narrative of the paper is that new mimetic alleles have arisen and spread to high frequency, and their dominance over the pre-existing alleles is consistent with Haldane's sieve. It would therefore make sense to test for selective sweep signatures within each morph (and its corresponding dsx haplotype), rather than at the species level. This would allow a test of the prediction that those morphs that arose most recently would have the strongest sweep signatures.

Sweep signatures erode over time - see Figure 2 of Moest et al. 2020 (https://doi.org/10.1371/journal.pbio.3000597), and it is unclear whether we expect the signatures of the original sweeps of these haplotypes to still be detectable at all. Moest et al show that sweep signatures are completely eroded by 1N generations after the event, and probably not detectable much sooner than that, so assuming effective population sizes of these species of a few million, at what time scale can we expect to detect sweeps? If these putative sweeps are in fact more recent than the origin of the different morphs, perhaps they would more likely be associated with the refinement of mimicry, but not necessarily providing evidence for or against a Haldane's sieve process in the origin of the morphs.

Our original plan was to perform signatures of sweeps on individual morphs, but we have very small sample sizes for individual morphs in some species, which made it difficult to perform the analysis. We agree that signatures of selective sweeps cannot give us an estimate of possible timescales of the sweep. They simply indicate that there may have been a sweep in a certain genomic region. Therefore, with just the data from selective sweeps, we cannot determine whether these occurred with refining of mimicry or the mimetic phenotype itself. We have thus made no interpretations regarding time scales or causal events of the sweep. Additionally, we discuss the results we obtained for individual alleles represent what could have occurred at the point of origin of mimetic resemblance or in the course of perfecting the resemblance, although we cannot differentiate between the two at this point (lines 320 to 333).

(2) Selective sweep methods

A tool called RAiSD was used to detect signatures of selective sweeps, but this manuscript does not describe what signatures this tool considers (reduced diversity, skewed frequency spectrum, increased LD, all of the above?). Given the comment above, would this tool be sensitive to incomplete sweeps that affect only one morph in a species-level dataset? It is also not clear how RAiSD could identify signatures of selective sweeps at individual SNPs (line 206). Sweeps occur over tracts of the genome and it is often difficult to associate a sweep with a single gene.

RAiSD (https://www.nature.com/articles/s42003-018-0085-8) detects selective sweeps using the μ statistic, which is a combined score of SFS, LD, and genetic diversity along a chromosome. The tool is quite sensitive and is able to detect soft sweeps. RAiSD can use a VCF variant file comprising of SNP data as input and uses an SNP-driven sliding window approach to scan the genome for signatures of sweep. Using an SNP file instead of runs of sequences prevents repeated calculations in regions that are sparse in variants, thereby optimizing execution time. Due to the nature of the input we used, the μ statistic was also calculated per site. We then tried to annotate the SNPs based on which genes they occur in and found that all species showing mimicry had atleast one site that showed a signature of sweep contained within the dsx locus.

(3) Episodic diversification

Very little information is provided about the Branch-site Unrestricted Statistical Test for Episodic Diversification (BUSTED) and Mixed Effects Model of Evolution (MEME), and what hypothesis the authors were testing by applying these methods. Although it is not mentioned in the manuscript, a quick search reveals that these are methods to study codon evolution along branches of a phylogeny. Without this information, it is difficult to understand the motivation for this analysis.

We thank you for bringing this to our notice, we will add a few lines in the Methods about the hypothesis we were testing and the motivation behind this analysis. We will additionally cite a previous study from our group which used these and other methods to study the molecular evolution of dsx across insect lineages.

(4) GWAS for form romulus

The authors argue that the lack of SNP associations within dsx for form romulus is caused by poor read mapping in the inverted region itself (line 125). If this is true, we would expect strong association in the regions immediately outside the inversion. From Figure S3, there are four discrete peaks of association, and the location of dsx and the inversion are not indicated, so it is difficult to understand the authors' interpretation in light of this figure.

We indeed observe the regions flanking dsx showing the highest association in our GWAS. This is a bit tricky to demonstrate in the figure as the genome is not assembled at the chromosome level. However, the association peaks occur on scf 908437033 at positions 2192979, 1181012 and 1352228 (Fig. S3c, Table S3) while dsx is located between 1938098 and 2045969. We will add the position of dsx in the figure legend of the revised manuscript.

(5) Form theseus

Since there appears to be only one sequence available for form theseus (actually it is said to be "P. javanus f. polytes/theseus"), is it reasonable to conclude that "the dsx coding sequence of f. theseus was identical to that of f. polytes in both P. javanus and P. alphenor" (Line 151)? Looking at the Clarke and Sheppard (1972) paper cited in the statement that "f. polytes and f. theseus show equal dominance" (line 153), it seems to me that their definition of theseus is quite different from that here. Without addressing this discrepancy, the results are difficult to interpret.

Among P. javanus individuals sampled by us, we obtained just one individual with f. theseus and the H P allele, however, in the data we added from a previously published study (Zhang et. al. 2017), we were able to add nine more individuals of this form (Fig. S4b and S7), while we did not show these individuals in Fig 3 (which was based on PCR amplification and sequencing of individual exons od dsx), all the analysis with sequence data was performed on 10 theseus individuals in total. In Zhang et. al. the authors observed what we now know are species specific differences when comparing theseus and polytes dsx alleles and not allele-specific differences. Our observations were consistent with these findings.

Welcome back.

This is part two of this lesson.

We're going to continue immediately from the end of part one.

So let's get started.

So focusing specifically on the animals for life scenario.

So what we're going to do in the upcoming demo lesson, to implement a truly resilient architecture for net services in a VPC, you need a net gateway in a public subnet inside each availability zone that the VPC uses.

So just like on the diagram that you've gone through now.

And then as a minimum, you need private route tables in each availability zone.

In this example, AZA, AZB, and then AZC.

Each of these would need to have their own route table, which would have a default IP version for route, which points at the net gateway in the same availability zone.

That way, if any availability zone fails, the others could continue operating without issues.

Now, this is important.

I've seen it in a few of some questions.

Where it suggests that one net gateway is enough, that a net gateway is truly regionally resilient.

This is false.

A net gateway is highly available in the availability zone that it's in.

So if hardware fails or it needs to scale to cope with load, it can do so in that AZ.

But if the whole AZ fails, there is no failover.

You provision a net gateway into a specific availability zone, not the region.

It's not like the internet gateway, which by default is region resilient.

For a net gateway, you have to deploy one into each AZ that you use if you need that region resilience.

Now, my apologies in advance for the small text.

It's far easier to have this all on screen at once.

I mentioned at the start of the lesson that net used to be provided by net instances, and these are just for the net process running on an EC2 instance.

Now, I don't expect this to feature on the exam at this point.

But if you ever need to use a net instance, by default, EC2 filters all traffic that it sends or receives.

It essentially drops any data that is on its network card when that network card is not either the source or the destination.

So if an instance is running as a net instance, then it will be receiving some data which the source address will be of other resources in that VPC.

And the destination will be a host on the internet.

So it will neither be the source nor the destination.

So by default, that traffic will be dropped.

And if you need to allow an EC2 instance to function as a net instance, then you need to disable a feature called source and destination checks.

This can be disabled via the console UI, the CLI, or the API.

The only reason I mention this is I have seen this question in the exam before, and if you do implement this in a real-world production-style scenario, you need to be aware that this feature exists.

I don't want you wasting your time trying to diagnose this feature.

So if you just right-click on an instance in the console, you'll be able to see an option to disable source and destination checks.

And that is required if you want to use an EC2 instance as a net instance.

Now, at the highest level, architecturally, net instances and net dayways are kind of the same.

They both need a public ID address.

They both need to run in a public subnet, and they both need a functional internet gateway.

But at this point, it's not really preferred to use EC2 running as a net instance.

It's much easier to use a net gateway, and it's recommended by AWS in most situations.

But there are a few key scenarios where you might want to consider using an EC2-based net instance.

So let's just step through some of the criteria that you might be looking at when deploying net services.

If you value availability, bandwidth, low levels of maintenance, and high performance, then you should use net gateways.

That goes for both real-world production usage, as well as being default for answering any exam questions.

A net gateway offers high-end performance, its scales, its custom design, perform network address translation.

A net instance in comparison is limited by the capabilities of the instances running on, and that instance is also general purpose, so it won't offer the same level of custom design performance as a net gateway.

Now, availability is another important consideration, and that instance is a single EC2 instance running inside an availability zone.

It will fail if the EC2 hardware fails.

It will fail if its storage fails or if its network fails, and it will fail if the AZ itself fails entirely.

A net gateway has some benefits over a net instance.

So inside one availability zone, it's highly available, so it can automatically recover, it can automatically scale.

So it removes almost all of the risks of outage versus a net instance.

But it will still fail entirely if the AZ fails entirely.

You still need provision, multiple net gateways, spread across all the AZs that you intend to use, if you want to ensure complete availability.

For maximum availability, a net gateway in every AZ you use.

This is critical to remember for the exam.

Now, if cost is your primary choice, if you're a financially challenged business, or if the VPC that you're deploying net services into is just a test VPC or something that's incredibly low volume, then a net instance can be cheaper.

It can also be significantly cheaper at high volumes of data.

You've got a couple of options.

You can use a very small EC2 instance, even ones that are free tier eligible to reduce costs, and the instances can also be fixed in size, meaning they offer predictable costs.

A net gateway will scale automatically, and you'll build for both the net gateway and the amount of data transferred, which increases as the gateway scales.

A net gateway is also not free tier eligible.

Now, this is really important because when we deploy these in the next demo lesson, it's one of those services that I need to warn you will come at a cost, so you need to be aware of that fact.

You will be charged for a net gateway regardless of how small the usage.

Net instances also offer other niche advantages because they're just EC2 instances.

You can connect to them just like you would any other EC2 instance.

You can multi-purpose them so you can use them for other things, such as passing hosts.

You can also use them for port forwarding, so you can have the port on the instance externally that could be connected to over the public internet, and have this forwarded-on for an instance inside the VPC.

Maybe port 8 if a web, or port 443 for secure web.

You can be completely flexible when you use net instances.

With a net gateway, this isn't possible because you don't have access to manage it.

It's a managed service.

Now, this comes up all the time in the exam, so try and get it really clear in your memory, and that gateway cannot be used as a passing host.

It cannot do port forwarding because you cannot connect to its operating system.

Now, finally, this is again one focus on the exam.

Net instances are just EC2 instances, so you can filter traffic using the network ACLs on the subnet instances in, or security groups directly associated with that instance.

Net gateways don't support security groups.

You can only use knuckles with net gateways.

This one comes up all the time in the exam, so it's worth noting down and maybe making a flashcard with.

Now, a few more things before we finish up.

What about IP version 6?

The focus of net is to allow private IP version 4 addresses to be used to connect in an outgoing only way to the AWS public zone and public internet.

Inside AWS, all IP version 6 addresses are publicly routable, so this means that you do not require net when using IP version 6.

The internet gateway works directly with IP version 6 addresses, so if you choose to make an instance in a private subnet, have a default IP version 6 route to the internet gateway, it will become a public instance.

As long as you don't have any knuckles or any security groups, any IP version 6 IP address in AWS can communicate directly with the AWS public zone and the public internet.

So the internet gateway can work directly with IP version 6.

Net gateways do not work with IP version 6, they're not required and they don't function with IP version 6.

So for the exam, if you see any questions which mention IP version 6 and net gateways, you can exclude the answer.

Net gateways do not work with IP version 6 and you can repeat it because I really wanted to stick in your memory.

So with any subnet inside AWS, which has been configured for IP version 6, if you add the IP version 6 default route, which is colon colon 4 slash 0, if you add that route and you point that route at the internet gateway as a target, that will give that instance bi-directional connectivity to the public internet and it will allow it to reach the AWS public zone and public services.

One service that we'll be talking about later on in the course when I cover more advanced features of VPC is a different type of gateway, known as an egress-only internet gateway.

This is a specific type of internet gateway that works only with IP version 6 and you use it when you want to give an IP version 6 instance outgoing only access to the public internet and the AWS public zone.

So don't worry, we'll be covering that later in the course, but I want to get it really burned into your memory that you do not use net and you do not use net gateways with IP version 6.

It will not work.

Now to get you some experience of using net gateways, it's time for a demo.

In the demo lesson, I'm going to be stepping you through what you need to do to provision a completely resilient net gateway architecture.

So that's using net gateway in each availability zone as well as configuring the routing required to make it work.

It's going to be one of the final pieces to our multi-tier VPC and it will allow private instances to have full outgoing internet access.

Now I can't wait for us to complete this together.

It's going to be a really interesting demo, one that will be really useful if you're doing this in the real world or if you have to answer exam questions related to net or net gateway.

So go ahead, complete the video and when you're ready, join me in the demo.

Author response:

The following is the authors’ response to the original reviews.

Reviewer #1 (Public Review):

Weaknesses:

The match between fractal and classical cycles is not one-to-one. For example, the fractal method identifies a correlation between age and cycle duration in adults that is not apparent with the classical method. This raises the question as to whether differences are due to one method being more reliable than another or whether they are also identifying different underlying biological differences. It is not clear for example whether the agreement between the two methods is better or worse than between two human scorers, which generally serve as a gold standard to validate novel methods. The authors provide some insight into differences between the methods that could account for differences in results. However, given that the fractal method is automatic it would be important to clearly identify criteria for recordings in which it will produce similar results to the classical method.

Thank you for these insightful suggestions. In the revised Manuscript, we have added a number of additional analyses that provide a quantitative comparison between the classical and fractal cycle approaches aiming to identify the source of the discrepancies between classical and fractal cycle durations. Likewise, we assessed the intra-fractal and intra-classical method reliability as outlined below.

Reviewer #1 (Recommendations For The Authors):

One of the challenges in interpreting the results of the manuscript is understanding whether the differences between the two methods are due to a genuine difference in what these two methods are quantifying or simply noise/variability in each method. If the authors could provide some more insight into this, it would be a great help in assessing their findings and I think bolster the applicability of their method.

(1) Method reliability: The manuscript clearly shows that cycle length is robustly correlated between fractal and classical in multiple datasets, however, it is hard to assign a meaningful interpretation to the correlation value (ie R = 0.5) without some reference point. This could be provided by looking at the intra-method correlation of cycle lengths. In the case of classical scoring, inter-scorer results could be compared, if the R-value here is significantly higher than 0.5 it would suggest genuine differences between the methods. In the case of fractal scoring, inter-electrode results could be compared / results with slight changes to the peak prominence threshold or smoothing window.

In the revised Manuscript, we performed the following analyses to show the intra-method reliability:

a) Classical cycle reliability: For the revised Manuscript, an additional scorer has independently defined classical sleep cycles for all datasets and marked sleep cycles with skipped REM sleep. Likewise, we have performed automatic sleep cycle detection using the R “SleepCycles” package by Blume & Cajochen (2021). We have added a new Table S8 to Supplementary Material 2 that shows the averaged cycle durations and cycle numbers obtained by the two human scorers and automatic algorithm as well as the inter-scorer rate agreement. We have added a new sheet named “Classical method reliability” that reports classical cycle durations for each participant and each dataset as defined by two human scorers and the algorithm To the Supplementary Excel file.

We found that the correlation coefficients between two human scorers ranged from 0.69 to 0.91 (in literature, r’s > 0.7 are defined as strong scores) in different datasets, thus being higher than correlation coefficients between fractal and classical cycle durations, which in turn ranged from 0.41 to 0.55 (r’s in the range of 0.3 – 0.7 are considered moderate scores). The correlation coefficients between human raters and the automatic algorithm showed remarkably lower coefficients ranging from 0.30 to 0.69 (moderate scores) in different datasets, thus lying within the range of the correlation coefficients between fractal and classical cycle durations. This analysis is reported in Supplementary Material 2, section ”Intra-classical method reliability” and Table S8.

b) Fractal cycle reliability: In the revised Supplementary Material 2 of our Manuscript, we assessed the intra-fractal method reliability, we correlated between the durations of fractal cycles calculated as defined in the main text, i.e., using a minimum peak prominence of 0.94 z and smoothing window of 101 thirty-second epochs, with those calculated using a minimum peak prominence ranging from 0.86 to 1.20 z with a step size of 0.04 z and smoothing windows ranging from 81 to 121 thirty-second epochs with a step size of 10 epochs (Table S7). We found that fractal cycle durations calculated using adjacent minimum peak prominence (i.e., those that differed by 0.04 z) showed r’s > 0.92, while those calculated using adjacent smoothing windows (i.e., those that differed by 10 epochs) showed r’s > 0.84. In addition, we correlated fractal cycle durations defined using different channels and found that the correlation coefficients ranged between 0.66 – 0.67 (Table S1). Thus, most of the correlations performed to assess intra-fractal method reliability showed correlation coefficients (r > 0.6) higher than those obtained to assess inter-method reliability (r = 0.41 – 0.55), i.e., correlations between fractal and classical cycle. This analysis is reported in Supplementary Material 2, section ”Intra-fractal method reliability” and Table S7. Likewise, we have added a new sheet named “Fractal method reliability” that reports the actual values for the abovementioned parameters to the Supplementary Excel file. For a discussion on potential sources of differences, see below.

(2) Origin of method differences: The authors outline a few possible sources of discrepancies between the two methods (peak vs REM end, skipped REM cycle detection...) but do not quantify these contributions. It would be interesting to identify some factors that could predict for either a given night of sleep or dataset whether it is likely to show a strong or weak agreement between methods. This could be achieved by correlating measures of the proposed differences ("peak flatness", fractal cycle depth, or proportion of skipped REM cycles) with the mismatch between the two methods.

In the revised Manuscript, we have quantified a few possible sources of discrepancies between the durations of fractal vs classical cycles and added a new section named “Sources of fractal and classical cycle mismatches” to the Results as well as new Tables 5 and S10 (Supplementary Material 2). Namely, we correlated the difference in classical vs fractal sleep cycle durations on the one side, and either the amplitude of fractal descent/ascent (to reflect fractal cycle depth), duration of cycles with skipped REM sleep/TST, duration of wake after sleep onset/TST or the REM episode length of a given cycle (to reflect peak flatness) on the other side. We found that a higher difference in classical vs fractal cycle duration was associated with a higher proportion of wake after sleep onset (r = 0.226, p = 0.001), shallower fractal descents (r = 0.15, p = 0.002) and longer REM episodes (r = 0.358, p < 0.001, n = 417 cycles, Table S10 in Supplementary Material 2). The rest of the assessed parameters showed no significant correlations (Table S10). We have added a new sheet named “Fractal-classical mismatch” that reports the actual values for the abovementioned parameters to the Supplementary Excel file.  

(3) Skipped REM cycles: the authors underline that the fractal method identified skipped REM cycles. It seems likely that manual identification of skipped REM cycles is particularly challenging (ie we would expect this to be a particular source of error between two human scorers). If this is indeed the case, it would be interesting to discuss, since it would highlight an advantage of their methodology that they already point out (l644).

In the revised Manuscript, we have added the inter-scorer rate agreement regarding cycles with skipped REM sleep, which was equal to 61%, which is 32% lower than the performance of our fractal cycle algorithm (93%). These findings are now reported in the “Skipped cycles” section of the Results and in Table S9 of Supplementary Material 2. We also discuss them in Discussion:

“Our algorithm detected skipped cycles in 93% of cases while the hypnogram-based agreement on the presence/absence of skipped cycles between two independent human raters was 61% only; thus, 32% lower. We deduce that the fractal cycle algorithm detected skipped cycles since a lightening of sleep that replaces a REM episode in skipped cycles is often expressed as a local peak in fractal time series.”<br /> Discussion, section “Fractal and classical cycles comparison”, paragraph 5.

Minor comments:

- In the subjects where the number of fractal and classical cycles did not match, how large was the difference (ie just one extra cycle or more)? Correlating cycle numbers could be one way to quantify this.

In the revised Manuscript, we have reported the required information for the participants with no one-to-one match (46% of all participants) as follows: 

“In the remaining 46% of the participants, the difference between the fractal and classical cycle numbers ranged from -2 to 2 with the average of -0.23 ± 1.23 cycle. This subgroup had 4.6 ± 1.2 fractal cycles per participant, while the number of classical cycles was 4.9 ± 0.7 cycles per participant. The correlation coefficient between the fractal and classical cycle numbers was 0.280 (p = 0.006) and between the cycle durations – 0.278 (p=0.006).” Results, section “Correspondence between fractal and classical cycles”, last paragraph.

- When discussing the skipped REM cycles (l467), the authors explain: "For simplicity and between-subject consistency, we included in the analysis only the first cycles". I'm not sure I understood this, could they clarify to which analysis they are referring to?

In the revised Manuscript, we performed this analysis twice: using first cycles and using all cycles and therefore have rephrased this as follows:

_“We tested whether the fractal cycle algorithm can detect skipped cycles, i.e., the cycles where an anticipated REM episode is skipped (possibly due to too high homeostatic pressure). We performed this analysis twice. First, we counted all skipped cycles (except the last cycles of a night, which might lack REM episode for other reasons, e.g., a participant had/was woken up). Second, we counted only the first classical cycles (i.e., the first cycle out of the 4 – 6 cycles that each participant had per night, Fig. 3 A – B) as these cy_cles coincide with the highest NREM pressure. An additional reason to disregard skipped cycles observed later during the night was our aim to achieve higher between-subject consistency as later skipped cycles were observed in only a small number of participants.” Results, section “Skipped cycles”, first paragraph.

- The inclusion of all the hypnograms as a supplementary is a great idea to give the reader concrete intuition of the data. If the limits of the sleep cycles for both methods could be added it would be very useful.

Supplementary Material 1 has been updated such that each graph has a mark showing the onsets of fractal and classical sleep cycles, including classical cycles with skipped REM sleep.

- The difference in cycle duration between adults and children seems stronger / more reliable for the fractal cycle method, particularly in the histogram (Figure 3C). Is this difference statistically significant?

In the revised Manuscript, we have added the Multivariate Analysis of Variance to compare F-values, partial R-squared and eta squared. The findings are as follows:

“To compare the fractal approach with the classical one, we performed a Multivariate Analysis of Variance with fractal and classical cycle durations as dependent variables, the group as an independent variable and the age as a covariate. We found that fractal cycle durations showed higher F-values (F(1, 43)  \= 4.5 vs F(1, 43) = 3.1), adjusted R squared (0.138 vs 0.089) and effect sizes (partial eta squared 0.18 vs 0.13) than classical cycle durations.” Results, Fractal cycles in children and adolescents, paragraph 3.

There have been some recent efforts to define sleep cycles in an automatic way using machine learning approaches. It could be interesting to mention these in the discussion and highlight their relevance to the general endeavour of automatizing the sleep cycle identification process.

In the Discussion of the revised Manuscript, we have added the section on the existing automatic sleep cycle definition algorithms:

“Even though recently, there has been a significant surge in sleep analysis incorporating various machine learning techniques and deep neural network architectures, we should stress that this research line mainly focused on the automatic classification of sleep stages and disorders almost ignoring the area of sleep cycles. Here, as a reference method, we used one of the very few available algorithms for sleep cycle detection (Blume & Cajochen, 2021). We found that automatically identified classical sleep cycles only moderately correlated with those detected by human raters (r’s = 0.3 – 0.7 in different datasets). These coefficients lay within the range of the coefficients between fractal and classical cycle durations (r = 0.41 – 0.55, moderate) and outside the range of the coefficients between classical cycle durations detected by two human scorers (r’s = 0.7 – 0.9, strong, Supplementary Material 2, Table S8).” Discussion, section “Fractal and classical cycles comparison”, paragraph 4.

Reviewer #2 (Public Review):

One weakness of the study, from my perspective, was that the IRASA fits to the data (e.g. the PSD, such as in Figure 1B), were not illustrated. One cannot get a sense of whether or not the algorithm is based entirely on the fractal component or whether the oscillatory component of the PSD also influences the slope calculations. This should be better illustrated, but I assume the fits are quite good.

Thank you for this suggestion. In the revised Manuscript, we have added a new figure (Fig.S1 E, Supplementary Material 2), illustrating the goodness of fit of the data as assessed by the IRASA method.

The cycles detected using IRASA are called fractal cycles. I appreciate the use of a simple term for this, but I am also concerned whether it could be potentially misleading? The term suggests there is something fractal about the cycle, whereas it's really just that the fractal component of the PSD is used to detect the cycle. A more appropriate term could be "fractal-detected cycles" or "fractal-based cycle" perhaps?

We agree that these cycles are not fractal per se. In the Introduction, when we mention them for the first time, we name them “fractal activity-based cycles of sleep” and immediately after that add “or fractal cycles for short”. In the revised version, we renewed this abbreviation with each new major section and in Abstract. Nevertheless, given that the term “fractal cycles” is used 88 times, after those “reminders”, we used the short name again to facilitate readability. We hope that this will highlight that the cycles are not fractal per se and thus reduce the possible confusion while keeping the manuscript short.

The study performs various comparisons of the durations of sleep cycles evaluated by the IRASA-based algorithm vs. conventional sleep scoring. One concern I had was that it appears cycles were simply identified by their order (first, second, etc.) but were not otherwise matched. This is problematic because, as evident from examples such as Figure 3B, sometimes one cycle conventionally scored is matched onto two fractal-based cycles. In the case of the Figure 3B example, it would be more appropriate to compare the duration of conventional cycle 5 vs. fractal cycle 7, rather than 5 vs. 5, as it appears is currently being performed.

In cases where the number of fractal cycles differed from the number of classical cycles (from 34 to 55% in different datasets as in the case of Fig.3B), we did not perform one-to-one matching of cycles. Instead, we averaged the duration of the fractal and classical cycles over each participant and only then correlated between them (Fig.2C). For a subset of the participants (45 – 66% of the participants in different datasets) with a one-to-one match between the fractal and classical cycles, we performed an additional correlation without averaging, i.e., we correlated the durations of individual fractal and classical cycles (Fig.4S of Supplementary Material 2). This is stated in the Methods, section Statistical analysis, paragraph 2.

There are a few statements in the discussion that I felt were either not well-supported. L629: about the "little biological foundation" of categorical definitions, e.g. for REM sleep or wake? I cannot agree with this statement as written. Also about "the gradual nature of typical biological processes". Surely the action potential is not gradual and there are many other examples of all-or-none biological events.

In the revised Manuscript, we have removed these statements from both Introduction and Discussion.

The authors appear to acknowledge a key point, which is that their methods do not discriminate between awake and REM periods. Thus their algorithm essentially detected cycles of slow-wave sleep alternating with wake/REM. Judging by the examples provided this appears to account for both the correspondence between fractal-based and conventional cycles, as well as their disagreements during the early part of the sleep cycle. While this point is acknowledged in the discussion section around L686. I am surprised that the authors then argue against this correspondence on L695. I did not find the "not-a-number" controls to be convincing. No examples were provided of such cycles, and it's hard to understand how positive z-values of the slopes are possible without the presence of some wake unless N1 stages are sufficient to provide a detected cycle (in which case, then the argument still holds except that its alterations between slow-wave sleep and N1 that could be what drives the detection).

In the revised Manuscript, we have removed the “NaN analysis” from both Results and Discussion. We have replaced it with the correlation between the difference between the durations of the classical and fractal cycles and proportion of wake after sleep onset. The finding is as follows:

“A larger difference between the durations of the classical and fractal cycles was associated with a higher proportion of wake after sleep onset in 3/5 datasets as well as in the merged dataset (Supplementary Material 2, Table S10).” Results, section “Fractal cycles and wake after sleep onset”, last two sentences. This is also discussed in Discussion, section “Fractal cycles and age”, paragraph 1, last sentence. 

To me, it seems important to make clear whether the paper is proposing a different definition of cycles that could be easily detected without considering fractals or spectral slopes, but simply adjusting what one calls the onset/offset of a cycle, or whether there is something fundamentally important about measuring the PSD slope. The paper seems to be suggesting the latter but my sense from the results is that it's rather the former.

Thank you for this important comment. Overall, our paper suggests that the fractal approach might reflect the cycling nature of sleep in a more precise and sensitive way than classical hypnograms. Importantly, neither fractal nor classical methods can shed light on the mechanism underlying sleep cycle generation due to their correlational approach. Despite this, the advantages of fractal over classical methods mentioned in our Manuscript are as follows:

(1) Fractal cycles are based on a real-valued metric with known neurophysiological functional significance, which introduces a biological foundation and a more gradual impression of nocturnal changes compared to the abrupt changes that are inherent to hypnograms that use a rather arbitrary assigned categorical value (e.g., wake=0, REM=-1, N1=-2, N2=-3 and SWS=-4, Fig.2 A).

(2) Fractal cycle computation is automatic and thus objective, whereas classical sleep cycle detection is usually based on the visual inspection of hypnograms, which is time-consuming, subjective and error-prone. Few automatic algorithms are available for sleep cycle detection, which only moderately correlated with classical cycles detected by human raters (r’s = 0.3 – 0.7 in different datasets here).

(3) Defining the precise end of a classical sleep cycle with skipped REM sleep that is common in children, adolescents and young adults using a hypnogram is often difficult and arbitrary.   The fractal cycle algorithm could detect such cycles in 93% of cases while the hypnogram-based agreement on the presence/absence of skipped cycles between two independent human raters was 61% only; thus, 32% lower.

(4) The fractal analysis showed a stronger effect size, higher F-value and R-squared than the classical analysis for the cycle duration comparison in children and adolescents vs young adults. The first and second fractal cycles were significantly shorter in the pediatric compared to the adult group, whereas the classical approach could not detect this difference.

(5) Fractal – but not classical – cycle durations correlated with the age of adult participants.

These bullets are now summarized in Table 5 that has been added to the Discussion of the revised manuscript.

Reviewer #1 (Public review):

Summary:

This paper examines plasticity in early cortical (V1-V3) areas in an impressively large number of rod monochromats (individuals with achromatopia). The paper examines three things:

(1) Cortical thickness. It is now well established that early complete blindness leads to increases in cortical thickness. This paper shows increased thickness confined to the foveal projection zone within achromats. This paper replicates the work by Molz (2022) and Lowndes (2021), but the detailed mapping of cortical thickness as a function of eccentricity and the inclusion of higher visual areas is particularly elegant.

(2) Failure to show largescale reorganization of early visual areas using retinotopic mapping. This is a replication of a very recent study by Molz et al. but I believe, given anatomical variability (and the very large n in this study) and how susceptible pRF findings are to small changes in procedure, this replication is also of interest.

(3) Connective field modelling, examining the connections between V3-V1. The paper finds changes in the pattern of connections, and smaller connective fields in individuals with achromatopsia than normally sighted controls, and suggests that these reflect compensatory plasticity, with V3 compensating for the lower resolution V1 signal in individuals with achromatopsia.

Strengths:

This is a carefully done study (both in terms of data collection and analysis) that is an impressive amount of work. I have a number of methodological comments but I hope they will be considered as constructive engagement - this work is highly technical with a large number of factors to consider.

Weaknesses:

(1) Effects of eye-movements

I have some concerns with how the effects of eye-movements are being examined. There are two main reasons the authors give for excluding eye-movements as a factor in their results. Both explanations have limitations.

a) The first is that R2 values are similar across groups in the foveal confluence. This is fine as far as it goes, but R2 values are going to be low in that region. So this shows that eye-movements don't affect coverage (the number of voxels that generate a reliable pRF), but doesn't show that eye-movements aren't impacting their other measures.

b) The authors don't see a clear relationship between coverage and fixation stability. This seems to rest on a few ad hoc examples. (What happens if one plots mean fixation deviation vs. coverage (and sets the individuals who could not be calibrated as the highest value of calibrated fixation deviation. Does a relationship then emerge?).

In any case, I wouldn't expect coverage to be particularly susceptible to eye-movements. If a voxel in the cortex entirely projects to the scotoma then it should be robustly silent. The effects of eye-movements will be to distort the size and eccentricity estimates of voxels that are not entirely silent.

There are many places in the paper where eye-movements might be playing an important role.

Examples include the larger pRF sizes observed in achromats. Are those related to fixation instability? Given that fixation instability is expected to increase pRF size by a fixed amount, that would explain why ratios are close to 1 in V3 (Figure 4).

(2) Topography

The claim of no change in topography is a little confusing given that you do see a change in eccentricity mapping in achromats.

Either this result is real, in which case there *is* a change in topography, albeit subtle, or it's an artifact.

Perhaps these results need a little bit of additional scrutiny.

One reason for concern is that you see different functions relating eccentricity to V1 segments depending on the stimulus. That almost certainly reflects biases in the modelling, not reorganization - the curves of Figure 2D are exactly what Binda et al. predict.

Another reason for concern is that I'm very surprised that you see so little effect of including/not including the scotoma - the differences seem more like what I'd expect from simply repeating the same code twice. (The quickest sanity check is just to increase the size of the estimated scotoma to be even bigger?).

I'd also look at voxels that pass an R2>0.2 threshold for both the non-selective and selective stimulus. Are the pRF sizes the same for both stimuli? Are the eccentricity estimates? If not, that's another clear warning sign.

(3) Connective field modelling

Let's imagine a voxel on the edge of the scotoma. It will tend to have a connective field that borders the scotoma, and will be reduced in size (since it will likely exclude the cortical region of V1 that is solely driven by resting state activity). This predicts your rod monochromat data. The interesting question is why this doesn't happen for controls. One possibility is that there is top-down 'predictive' activity that smooths out the border of the scotoma (there's some hint of that in the data), e.g., Masuda and Wandell.

One thing that concerns me is that the smaller connective fields don't make sense intuitively. When there is a visual stimulus, connective fields are predominantly driven by the visual signal. In achromats, there is a large swath of cortex (between 1-2.5 degrees) which shows relatively flat tuning as regards eccentricity. The curves for controls are much steeper, See Figure 2b. This predicts that visually driven connective fields should be larger for achromats. So, what's going on? The beta parameter is not described (and I believe it can alter connective field sizes). Similarly, it's possible to get very small connective fields, but there wasn't a minimum size described in the thresholding. I might be missing something obvious, but I'm just deeply confused as to how the visual maps and the connectome maps can provide contradictory results given that the connectome maps are predominantly determined by the visual signal. Some intuition would be helpful.

Some analyses might also help provide the reader with insight. For example, doing analyses separately on V3 voxels that project entirely to scotoma regions, project entirely to stimulus-driven regions, and V3 voxels that project to 'mixed' regions.

The finding that pRF sizes are larger in achromats by a constant factor as a function of eccentricity is what differences in eye-movements would predict. It would be worth examining the relationship between pRF sizes and fixation stability.

This statement made me think how it is something that I have thought about a lot in my subconscious mind to tackle situations, such as how I have been misunderstood over time greatly just because I typed it out. The other person did not hear me say it so, he/she misinterpreted it so I had to explain myself and it would be easy to understand through my tone. It's elementary to misjudge someone's perspective when it's typed out.

Answering these questions I believe that for age I would use an integer with a constraint, for Name I would store it as a string that includes character constraints so it could handle all names but no one can type long random things. For address, I would use a string with structured fields so that someone can type in their street, city, state, and etc. For relationship status, I would use predefined options such as married, single, complicated so that someone can just chose out of the options so it makes it consistent and easy.

I thought that this exercise was so interesting and that even though some of the options were not necessarily wrong, some just felt more natural to choose. I asked my friends around me what they would say without telling them my answer and we all chose the same thing. I think it's weird but cool that there's this unspoken way about how as native English speakers we order the adjectives.

I think this is the hard part of reality that students will hide their true colors so they won't get judged from reality or have their school life be based on that. Students come to school for it to be a safe place and feel comfortable or put on a fake persona because the moment they go home they go back to reality. It's hard living a double life especially when you're able to see others do things or get things you've always wanted. I can say that i've experienced the we don't have enough for that and just shop for necessities, but i've been able to see others not being able to afford other things. It is a sad reality and wishing we can help everyone.

It's interesting to that white families would just leave and move away because of desegregation. It also shows the money and privledge they had to be able to make this big of a change as a result. I'm sure many of the white families leave bcause they didn't agree with schools being desegregated, but I wonder how many families left because they saw that the school system was struggling and in conflict, which would affect their children's education.

I agree, it’s clear that income plays a huge role in a child’s development. Families with more resources can provide better living conditions, access to quality education and extracurricular activities that enrich learning. Studies have shown that children who participate in these activities tend perform better in school. On the other hand, lower-income families often face significant barriers, not just in education but in overall well-being, which can hinder a child's potential. Addressing these disparities is essential for creating a more equitable society.

do you believe in the strength of academia, and if there were a "thing between the electorate and righteousness" what would it be?

what replaces the electoral college?

https://en.wikipedia.org/wiki/United_States_Electoral_College#:~:text=In%20the%20United%20States,%20the%20Electoral

When the Knights of the Round Table, or the Templars; or the Holy Guard stand up in this place and "Friday the 13th" happeneth to them all, in history and; I mean, did it just pass ... what kind of power stands between the end of "democracy" and it's very beginning?

Babylon, alas--at the very crawling of the vote; at the exact moment freedom could have "rung free" ...

free at last, free at last; then God almighty ...

Edom, where are we?

echoes of "Michele, and Kelly smart:" https://en.wikipedia.org/wiki/Extinction_event#:~:text=An%20extinction%20event%20(also%20known%20as

straight out of the day "we see your bridge" in D.C. echoed of ... something like the Manhattan picture of the street below Bianca's Empire State building; somewhere in South Florida; for some reason, somehow, something just like today.

Does it make a difference?

https://poets.org/poem/hamlet-act-iii-scene-i-be-or-not-be#:~:text=1616.%20To%20be,%20or%20not%20to

https://www.youtube.com/watch?v=23hZQiMoB7A

It's fascinating that if China just kept the invention of gunpowder to themselves, they could have been able to defeat anyone that crossed their paths. But instead, they didn't and now gunpowder is still used to this day for a variety of things.

I listen to music all the time and I never knew that this is how sound waves are created. It's crazy how all the sound information is just saved onto a number.

This made me think about how in different languages they have different phrases/words for the sounds that animals make, something that I find really interesting. For example, if I was trying to communicate this way with someone from a Spanish speaking country and I said "bark" trying to explain that I'm looking for a dog, they might be confused because they would say "guau" which means bark/woof. It's just really interesting to me how the sounds that animals make, which is generally universal, are different across languages.

doesnt sound like you are fun/approchable?

Welcome back.

In this lesson, I'll be talking about Network Address Translation, or NAT, a process of giving a private resource outgoing only access to the internet.

And a NAT gateway is the AWS implementation that's available within WPC.

There's quite a bit of theory to cover, so let's get started.

So what is NAT?

Well, it stands for Network Address Translation.

This is one of those terms which means more than people think that it does.

In a strict sense, it's a set of different processes which can adjust ID packets by changing their source or destination IP addresses.

Now, you've seen a form of this already.

The internet gateway actually performs a type of NAT known as static NAT.

It's how a resource can be allocated with a public IP version for address, and then when the packets of data leave those resources and pass through the internet gateway, it adjusts the source IP address on the packet from the private address to the public, and then sends the packet on, and then when the packet returns, it adjusts the destination address from the public IP address to the original private address.

That's called static NAT, and that's how the internet gateway implements public IP version for addressing.

Now, what most people think of when they think of NAT is a subset of NAT called IP Masquerading.

And IP Masquerading hides a whole private side IP block behind a single public IP.

So rather than the one private IP to one public IP process that the internet gateway does, NAT is many private IPs to one single IP.

And this technique is popular because IP version 4 addresses are running out.

The public address space is rapidly becoming exhausted.

IP Masquerading, or what we'll refer to for the rest of this lesson as NAT, gives a whole private range of IP addresses outgoing only access to the public internet and the AWS public zone.

I've highlighted outgoing because that's the most important part, because many private IPs use a single public IP.

Incoming access doesn't work.

Private devices that use NAT can initiate outgoing connections to internet or AWS public space services, and those connections can receive response data, but you cannot initiate connections from the public internet to these private IP addresses when NAT is used.

It doesn't work that way.

Now, AWS has two ways that it can provide NAT services.

Historically, you could use an EC2 instance configured to provide NAT, but it's also a managed service, the NAT gateway, which you can provision in the VPC to provide the same functionality.

So let's look at how this works architecturally.

This is a simplified version of the Animals for Life architecture that we've been using so far.

On the left is an application tier subnet in blue, and it's using the IP range 10.16.32.0/20.

So this is a private only subnet.

Inside it are three instances, I01, which is using the IP 10.16.32.10, I02, which is using 32.20, and I03, which is using 32.30.

These IP addresses are private, so they're not publicly routable.

They cannot communicate with the public internet or the AWS public zone services.

These addresses cannot be routed across a public style network.

Now, if we wanted this to be allowed, if we wanted these instances to perform certain activities using public networking, for example, software updates, how would we do it?

Well, we could make the subnet's public in the same way that we've done with the public subnets or the web subnets, but we might not want to do that architecturally.

With this multi-tier architecture that we're implementing together, part of the design logic is to have tiers which aren't public and aren't accessible from the public internet.

Now, we could also host some kind of software update server inside the VPC, and some businesses choose to do that.

Some businesses run Windows update services, all Linux update services inside their private network, but that comes with an admin overhead.

NAT offers us a third option, and it works really well in this style of situation.

We provision a NAT gateway into a public subnet, and remember, the public subnet allows us to use public IP addresses.

The public subnet has a route table attached to it, which provides default IP version 4 routes pointing at the internet gateway.

So, because the NAT gateway is located in this public web subnet, it has a public IP which is routable across the public internet, so it's now able to send data out and get data back in return.

Now, the private subnet where the instances are located can also have its own route table, and this route table can be different than the public subnet route table.

So, we could configure it so that the route table that's on the application subnet has a default IP version 4 route, but this time, instead of pointing at the internet gateway, like the web subnet users, we configure this private route table so that it points at the NAT gateway.

This means when those instances are sending any data to any IP addresses that do not belong inside the VPC, by default, this default route will be used, and that traffic will get sent to the NAT gateway.

So, let's have a look at how this packet flow works.

Let's simulate the flow packets from one of the private instances and see what the NAT gateway actually does.

So, first, instance 1 generates some data.

Let's assume that it's looking for software updates.

So, this packet has a source IP address of instance 1's private IP and a destination of 1.3.3.7.

For this example, let's assume that that's a software update server.

Now, because we have this default route on the route table of the application subnet, that packet is routed through to the NAT gateway.

The NAT gateway makes a record of the data packet.

It stores the destination that the packet is for, the source address of the instance sending it, and other details which help it identify the specific communication in future.

Remember, multiple instances can be communicating at once, and for each instance, it could be having multiple conversations with different public internet hosts.

So, the NAT gateway needs to be able to uniquely identify those.

So, it records the IP addresses involved, the source and destination, the port numbers, everything it needs, into a translation table.

So, the NAT gateway maintains something called a translation table which records all of this information.

And then, it adjusts the packet to the one that's been sent by the instance, and it changes the source address of this IP packet to be its own source address.

Now, if this NAT appliance were anywhere for AWS, what it would do right now is adjust the packet with a public routable address. - Hi. - Let's do this directly.

But remember, all the inside of the IPC really has directly attached to it a public IP version 4 address.

That's what the internet gateway does.

So, the NAT gateway, because it's in the web subnet, it has a default route, and this default route points at the internet gateway.

And so, the packet is moved from the NAT gateway to the internet gateway by the IPC router.

At this point, the internet gateway knows that this packet is from the NAT gateway.

It knows that the NAT gateway has a public IP version 4 address associated with it, and so, it modifies the packet to have a source address of the NAT gateway's public address, and it sends it on its way.

The NAT gateway's job is to allow multiple private IP addresses to masquerade behind the IP address that it has.

That's where the term IP masquerading comes from.

That's why it's more accurate.

So, the NAT gateway takes all of the incoming packets from all of the instances that it's managing, and it records all the information about the communication.

It takes those packets, it changes the source address from being those instances to its own IP address, its own external-facing IP address.

If it was outside AWS, this would be a public address directly.

That's how your internet router works for your home network.

All of the devices internally on your network talk out using one external IP address, your home router uses NAT.

But because it's in AWS, it doesn't have directly attached a real public IP.

The internet gateway translates from its IP address to the associated public one.

So, that's how the flow works.

If you need to give an instance its own public IP version for address, then only the internet gateway is required.

If you want to give private instances outgoing access to the internet and the AWS public services such as S3, then you need both the NAT gateway to do this many-to-one translation and the internet gateway to translate from the IP of the NAT gateway to a real public IP version for address.

Now, let's quickly run through some of the key facts for the NAT gateway product that you'll be implementing in the next demo lesson.

First, and I hope this is logical for you by now, it needs to run from a public subnet because it needs to be able to be assigned a public IP version for IP address for itself.

So, to deploy a NAT gateway, you already need your VPC in a position where it has public subnets.

And for that, you need an internet gateway, subnets configured to allocate public IP version for addresses and default routes for those subnets pointing at the internet gateway.

Now, a NAT gateway actually uses a special type of public IP version for address that we haven't covered yet called an elastic IP.

For now, just know that these are IP version for addresses, which is static.

They don't change.

These IP addresses are allocated to your account in a region and they can be used for whatever you want until you reallocate them.

And NAT gateways use these elastic IPs, the one service which utilizes elastic IPs.

Now, they're talking about elastic IPs later on in the course.

Now, NAT gateways are an AZ resilient service.

If you read the AWS documentation, you might get the impression that they're fully resilient in a region like an internet gateway.

They're not, they're resilient in the AZ that they're in.

So they can recover from hardware failure inside an AZ.

But if an AZ entirely fails, then the NAT gateway will also fail.

For a fully region resilient service, so to mirror the high availability provided by an internet gateway, then you need to deploy one NAT gateway in each AZ that you're using in the VPC and then have a route table for private subnets in that availability zone, pointing at the NAT gateway also in that availability zone.

So for every availability zone that you use, you need one NAT gateway and one route table pointing at that NAT gateway.

Now, they aren't super expensive, but it can get costly if you have lots of availability zones, which is why it's important to always think about your VPC design.

Now, NAT gateways are a managed service.

You deploy them and AWS handle everything else.

They can scale to 45 gigabits per second in bandwidth and you can always deploy multiple NAT gateways and split your subnets across multiple provision products.

If you need more bandwidth, you can just deploy more NAT gateways.

For example, you could split heavy consumers across two different subnets in the same AZ, have two NAT gateways in the same AZ and just route each of those subnets to a different NAT gateway and that would quickly allow you to double your available bandwidth.

With NAT gateways, you'll build based on the number that you have.

So there's a standard hourly charge for running a NAT gateway and this is obviously subject to change in a different region, but it's currently about four cents per hour.

And note, this is actually an hourly charge.

So partial hours are billed as full hours.

And there's also a data processing charge.

So that's the same amount as the hourly charge around four cents currently per gigabyte of processed data.

So you've got this base charge that a NAT gateway consumes while running plus a charge based on the amount of data that you process.

So keep both of those things in mind for any NAT gateway related questions in the exam.

Don't focus on the actual values, just focus on the fact they have two charging elements.

Okay, so this is the end of part one of this lesson.

It's getting a little bit on the long side, and so I wanted to add a break.

It's an opportunity just to take a rest or grab a coffee.

Part two will be continuing immediately from the end of part one.

So go ahead, complete the video, and when you're ready, join me in part two.

Welcome back.

In this lesson I want to talk in detail about security groups within AWS.

These are the second type of security filtering feature commonly used within AWS, the other type being network access control lists which we've previously discussed.

So security groups and knuckles share many broad concepts but the way they operate is very different and it's essential that you understand those differences and the features offered by security groups for both the exam and real-world usage.

So let's just jump in and get started.

In the lesson on network access control lists I explained that they're stateless and by now you know what stateless and stateful mean.

Security groups are stateful, they detect response traffic automatically for a given request and this means that if you allow an inbound or outbound request then the response is automatically allowed.

You don't have to worry about configuring ephemeral ports, it's all handled by the product.

If you have a web server operating on TCP 443 and you want to allow access from the public internet then you'll add an inbound security group rule allowing inbound traffic on TCP 443 and the response which is using ephemeral ports is automatically allowed.

Now security groups do have a major limitation and that's that there is no explicit deny.

You can use them to allow traffic or you can use them to not allow traffic and this is known as an implicit deny.

So if you don't explicitly allow traffic then you're implicitly denying it but you can't and this is important you're unable to explicitly deny traffic using security groups and this means that they can't be used to block specific bad actors.

Imagine you allow all source IP addresses to connect to an instance on port 443 but then you discover a single bad actor is attempting to exploit your web server.

Well you can't use security groups to block that one specific IP address or that one specific range.

If you allow an IP or if you allow an IP range or even if you allow all IP addresses then security groups cannot be used to deny a subset of those and that's why typically you'll use network access control lists in conjunction with security groups where the knuckles are used to add explicit denies.

Now security groups operate above knuckles on the OSI7 layer stack which means that they have more features.

They support IP and side-based rules but they also allow referencing AWS logical resources.

This includes all the security groups and even itself within rules.

I'll be covering exactly how this works on the next few screens.

Just know at this stage that it enables some really advanced functionality.

An important thing to understand is that security groups are not attached to instances nor are they attached to subnet.

They're actually attached to specific elastic network interfaces known as ENIs.

Now even if you see the user interface present this as being able to attach a security group to an instance know that this isn't what happens.

When you attach a security group to an instance what it's actually doing is attaching the security group to the primary network interface of that instance.

So remember security groups are attached to network interfaces that's an important one to remember for the exam.

Now at this point let's step through some of the unique features of security groups and it's probably better to do this visually.

Let's start with a public subnet containing an easy to instance and this instance has an attached primary elastic network interface.

On the right side we have a customer Bob and Bob is accessing the instance using HDTBS so this means TCP but 443.

Conceptually think of security groups as something which surrounds network interfaces in this case the primary interface of the EC2 instance.

Now this is how a typical security group might look.

It has inbound and outbound rules just like a network ACL and this particular example is showing the inbound rules allowing TCP port 443 to connect from any source.

The security group applies to all traffic which enters or leaves the network interface and because they're stateful in this particular case because we've allowed TCP port 443 as the request portion of the communication the corresponding response part the connection from the instance back to Bob is automatically allowed.

Now lastly I'm going to repeat this point several times throughout this lesson.

Security groups cannot explicitly block traffic.

This means with this example if you're allowing 0.0.0.0.0 to access the instance on port TCP port 443 and this means the whole IP version for internet then you can't block anything specific.

Imagine Bob is actually a bad actor.

Well in this situation security groups cannot be used to add protection.

You can't add an explicit deny for Bob's IP address.

That's not something that security groups are capable of.

Okay so that's the basics.

Now let's look at some of the advanced bits of security group functionality.

Security groups are capable of using logical references.

Let's step through how this works with a similar example to the one you just saw.

We start with a VPC containing a public web subnet and a private application subnet.

Inside the web subnet is the Categoram application web instance and inside the app subnet is the back-end application instance.

Both of these are protected by security groups.

We have A4L-web and A4L-app.

Traffic wise we have Bob accessing the web instance over port TCP port 443 and because this is the entry point to the application which logically has other users than just Bob we're allowing TCP port 443 from any IP version for address and this means we have a security group with an inbound rule set which looks like this.

In addition to this front-end traffic the web instance also needs to connect with the application instance and for this example let's say this is using TCP port 1337.

Our application is that good.

So how best to allow this communication?

Well we could just add the IP address of the web instance into the security group of the application instance or if you wanted to allow our application to scale and change IPs then we could add the side arrangers of the subnets instead of IP addresses.

So that's possible but it's not taking advantage of the extra functionality which security groups provide.

What we could do is reference the web security group within the application security group.

So this is an example of the application security group.

Notice that it allows TCP port 1337 inbound but it references as the source a logical resource the security group.

Now using a logical resource reference in this way means that the source reference of the A4L-web security group this actually references anything which has this security group associated with it.

So in this example any instances which have the A4L-web security group attached to them can connect to any instances which have the A4L-web security group attached to them using TCP port 1337.

So in essence this references this.

So this logical reference within the application security group references the web security group and anything which has the web security group attached to it.

Now this means we don't have to worry about IP addresses or side arrangers and it also has another benefit.

It scales really well.

So as additional instances are added to the application subnet and web subnet and as those instances are attached to the relevant security groups they're impacted by this logical referencing allowing anything defined within the security group to apply to any new instances automatically.

Now this is critical to understand so when you reference a security group from another security group what you're actually doing is referencing any resources which have that security group associated with them.

So this substantially reduces the admin overhead when you have multi-tiered applications and it also simplifies security management which means it's prone to less errors.

Now logical references provide even more functionality.

They allow self referencing.

Let's take this as an example a private subnet inside AWS with an ever-changing number of application instances.

Right now it's three but it might be three, thirty or one.

What we can do is create a security group like this.

This one allows incoming communications on port TCP 1337 from the web security group but it also has this rule which is a self-referential rule allowing all traffic.

What this means is that if it's attached to all of the instances then anything with this security group attached can receive communication so all traffic from this security group and this effectively means anything that also has this security group attached to it.

So it allows communications to occur to instances which have it attached from instances which have it attached.

It handles any IP changes automatically which is useful in these instances within an auto scaling group which is provisioning and terminating instances based on load on the system.

It also allows for simplified management of any intra-app communications.

An example of this might be Microsoft the main controllers or managing application high availability within clusters.

So this is everything I wanted to cover about security groups within AWS.

So there's a lot of functionality and intelligence that you gain by using security groups versus network ACLs but it's important that you understand that while network ACLs do allow you to explicitly deny traffic security groups don't and so generally you would use network ACLs to explicitly block any bad actors and use security groups to allow traffic to your VPC based resources.

You do this because security groups are capable of this logical resource referencing and that means AWS logical resources in security groups or even itself to allow this free flow of communications within a security group.

At this point that is everything I wanted to cover in this lesson so go ahead and complete the video and when you're ready I'll look forward to you joining me in the next lesson.

CONNECTION ANNOTATIONS

Just as in the Trott and Lee LLM article, word vectors are described as a matchmaking site for words. AI generated photos could be viewed in a similar way. Not every date will result in a soul mate, therefore every match is not ideal. I often find myself frustrated when typing and my cell phone will decide that it is smarter than me and knows exactly what I am going to say next. I become irate when I hit send before, I realize those changes have been made and I sound as though I only completed my elementary school education. The same can be send for these photos, when an AI model acquires many photos to compile one master image, thought to be exactly what you are looking for, only to discover that AI is so generous, that they have provided the subject of the photo with an extra arm.

1. Similar to how Trott and Lee LLM article phrased AI as being similar to "turning the water on, making sure it's coming out of the right faucet, and when it's not, running around to tighten and loosen all the valves until it does", AI generated photos (7:20) can create biases that then we must run around to resolve. The original information inputted to generate a photo is like the water. Once it is turned on and a photo is generated, but it's not the photo we were expecting, we then have to tweak and edit the information until we receive a desired outcome.

Reviewer #3 (Public review):

Summary:

In this manuscript, Yip and colleagues incorporated the pipette cleaning technique into their existing dual-patch robotic system, "the PatcherBot", to allow sequential patching of more cells for synaptic connection detection in living brain slices. During dual-patching, instead of retracting all two electrodes after each recording attempt, the system cleaned just one of the electrodes and reused it to obtain another recording while maintaining the other. With one new patch clamp recording attempt, new connections can be probed. By placing one pipette in front of the other in this way, one can "walk" across the tissue, termed "patch-walking." This application could allow for probing additional neurons to test the connectivity using the same pipette in the same preparation.

Strengths:

Compared to regular dual-patch recordings, this new approach could allow for probing more possible connections in brain slices with dual-patch recordings, thus having the potential to improve the efficiency of identifying synaptic connections

Weaknesses:

While this new approach offers the potential to increase efficiency, it has several limitations that could curtail its widespread use.

Loss of Morphological Information: Unlike traditional multi-patch recording, this approach likely loses all detailed morphology of each recorded neuron. This loss is significant because morphology can be crucial for cell type verification and understanding connectivity patterns by morphological cell type.

Spatial Restrictions: The robotic system appears primarily suited to probing connections between neurons with greater spatial separation (~100µm ISD). This means it may not reliably detect connections between neurons in close proximity, a potential drawback given that the connectivity is much higher between spatially close neurons. This limitation could help explain the low connectivity rate (5%) reported in the study.

Limited Applicability: While the approach might be valuable in specific research contexts, its overall applicability seems limited. It's important to consider scenarios where the trade-off between efficiency and specific questions that are asked.<br /> Scalability Challenges: Scaling this method beyond a two-pipette setup may be difficult. Additional pipettes would introduce significant technical and logistical complexities.

Welcome back and by now you should understand the difference between stateless and stateful security protection.

In this lesson I want to talk about one security feature of AWS VPCs and a little bit more depth and that's network access control lists known as knuckles.

Now we do have a lot to cover so let's jump in and get started.

A network access control list we thought of as a traditional firewall available within AWS VPCs so let's look at a visual example.

A subnet within an AWS VPC which has two EC2 instances A and B.

The first thing to understand and this is core to how knuckles work within AWS is that they are associated with subnets.

Every subnet has an associated network ACL and this filters data as it crosses the boundary of that subnet.

In practice this means any data coming into the subnet is affected and data leaving the subnet is affected.

But and this is super important to remember connections between things within that subnet such as between instance A and instance B in this example are not affected by network ACLs.

Each network ACL contains a number of rules, two sets of rules to be precise.

We have inbound rules and outbound rules.

Now inbound rules only affect data entering the subnet and outbound rules affect data leaving the subnet.

Remember from the previous lesson this isn't always matching directly to request and response.

A request can be either inbound or outbound as can a response.

These inbound and outbound rules are focused only on the direction of traffic not whether it's request or response.

In fact and I'll cover this very soon knuckles are stateless which means they don't know if traffic is request or response.

It's all about direction.

Now rules match the destination IP or IP range, destination port or port range together with the protocol and they can explicitly allow or explicitly deny traffic.

Remember this one network ACLs offer both explicit allows and explicit denies.

Now rules are processed in order.

First a network ACL determines if the inbound or outbound rules apply.

Then it starts from the lowest rule number.

It evaluates traffic against each individual rule until it finds a match.

Then that traffic is either allowed or denied based on that rule and then processing stops.

Now this is critical to understand because it means that if you have a deny rule and an allow rule which match the same traffic but if the deny rule comes first then the allow rule might never be processed.

Lastly there's a catch all showed by the asterisk in the rule number and this is an implicit deny.

If nothing else matches then traffic will be denied.

So those are the basics.

Next let's move on to some more complex elements of network ACLs.

Now I just mentioned that network ACLs are stateless and this means that rules are required for both the request and the response part of every communication.

You need individual rules for those so one inbound and one outbound.

Take this example a multi-tiered application running in a VPC.

We've got a web server in the middle and an application server on the left.

On the right we have a user Bob using a laptop and he's accessing the website.

So he makes a connection using HTTPS which is TCP port 443 and this is the request as you know by now and this is also going to mean that a response is required using the ephemeral port range.

This ephemeral port is chosen at random from the available range decided by the operating system on Bob's laptop.

Now to allow for this initial communication if we're using network ACLs then we'll need to have one associated with the web subnet and it will need rules in the inbound and outbound sections of that network ACL.

Notice how on the inbound rule set we have rule number 110 which allows connections from anywhere and this is signified by 0.0.0.0 through this network ACL and this is allowed as long as it's using TCP port 443.

So this is what allows the request from Bob into the web server.

We also have on the outbound rule set rule number 120 and this allows outbound traffic to anywhere again 0.0.0.0 as long as the protocol is TCP using the port range of 1.0.2.4 to 65.5.3.5 and this is the ephemeral port range which I mentioned in the previous lesson.

Now this is not amazingly secure but with stateless firewalls this is the only way.

Now we also have the implicit denies and this is denoted by the rules with the star in the rule number and this means that anything which doesn't match rule 110 or 120 is denied.

Now it's also worth mentioning while I do have rule 110 and 120 number differently the rule numbers are unique on inbound and outbound so we could have the single rule number 110 on both rule sets and that would be okay.

It's just easier to illustrate this if I use unique rule numbers for each of the different rule sets.

Now let's move on and increase the complexity little.

So we have the same architecture we have Bob on the right, the web subnet in the middle and the application subnet on the left.

You know now that because network ACLs are stateless each communication requires one request rule and one response rule.

This becomes more complex when you have a multi-tiered architecture which operates across multiple subnets and let's step through this to illustrate why.

Let's say the pop initiates a connection to the web server we know about this already because I just covered it.

If we have a network ACL around the web subnet we'll need an inbound rule on the web network ACL.

There's also going to be response traffic so this is going to use the ephemeral port range and this is going to need an outbound rule on that same web network ACL so this should make sense so far.

But also the web server might need to communicate with the app server using some application TCP port.

Now this is actually crossing two subnet boundaries the web at subnet boundary and the application subnet boundary so it's going to need an outbound rule on the web at subnet knuckle and also an inbound rule on the application subnet knuckle.

Then we have the response for that as well from the app server through to the web server and this is going to be using ephemeral ports but this also crosses two subnet boundaries it leaves the application subnet which will need an outbound rule on that knuckle and it enters the web subnet which will also need an inbound rule on that network ACL and what if each of those servers need software updates it will get even more complex really quickly.

You always have to be aware of these rule pairs the application port request and the ephemeral response for every single communication in some cases you're going to have multi-tier architecture and this might mean the communications go through different subnets.

If you need software updates this will need more if you use network address translation or NAT you might need more rules still.

You'll need to worry about this if you use network ACLs within a vpc for traffic to a vpc or traffic from a vpc or traffic between subnets inside that vpc.

When a vpc is created it's created with a default network ACL and this contains inbound and outbound rule sets which have the default implicit deny but also a capsule allow and this means that the net effect is that all traffic is allowed so the default within a vpc is that knuckles have no effect they aren't used this is designed [Music] I need to be beginner friendly and reduce admin overhead.

AWS prefer using security groups which I'll be covering soon.

If you create your own custom network ACLs though that's a different story.

Custom knuckles are created for a specific vpc and initially they're associated with no subnets.

They only have one rule on both the inbound and outbound rule sets which is the default deny and the result is that if you associate this custom network ACL with any subnets all traffic will be denied so be careful with this it's radically different behavior than the default network ACL created with a vpc.

Now this point I just want to cover some finishing key points which you need to be aware of for any real-world usage and when you're answering exam questions.

So network access controlists remember they're known as knuckles they are stateless so they view request and response as different things so you need to add rules both for the request and for the response.

A knuckle only affects data which is crossing the subnet boundary so communications between instances in the same subnet is not affected by a network ACL on that subnet.

Now this can mean that if you do have data crossing between subnets then you need to make sure that each knuckle on both of those subnets has the appropriate inbound and outbound rules so you end up with a situation where one connection can in theory need two rules on each knuckle if that connection is crossing two different subnet boundaries.

Now knuckles are able to explicitly allow traffic and explicitly deny and the deny is important because as you'll see when I talk about security groups this is a capability that you need to network ACLs.

So network ACLs allow you to block specific IPs or specific IP ranges which are associated with bad actors so they're a really good security feature when you need to block any traffic attempting to exploit your systems.

Now network ACLs are not aware of any logical resources they only allow you to use IPs and cyber ranges ports and protocols you cannot reference logical resources within AWS and knuckles can also not be assigned two logical resources they're only assigned to subnets within VPCs within AWS.

Now knuckles are very often used together with security groups such as mentioned to add the capability to explicitly deny bad IPs or bad networks so generally you would use security groups to allow traffic and you use knuckles to deny traffic and I'll talk about exactly how this works in the next lesson.

Now each subnet within a VPC has one knuckle associated with it it's either going to be the default network ACL for that VPC or a custom one which you create and associate.

A single knuckle though can be associated with many different subnets so while a subnet can only have one network ACL one network ACL can be associated with many different subnets.

Now this point that is everything that I wanted to cover about network ACLs for this lesson so go ahead complete the video and when you're ready I'll look forward to you joining me in the next lesson.

we all exist as (relative) experts within some domain/field or other, acting creatively within it (with others) -- not just folk in the creative industry!

(it's just that they have this specific focus...)

they found dead birds that fell down the chimney. It is also sad and painful.

Reviewer #1 (Public review):

Summary & Assessment:

The catalytic core of the eukaryotic decapping complex consists of the decapping enzyme DCP2 and its key activator DCP1. In humans, there are two paralogs of DCP1, DCP1a and DCP1b, that are known to interact with DCP2 and recruit additional cofactors or coactivators to the decapping complex; however, the mechanisms by which DCP1 activates decapping and the specific roles of DCP1a versus DCP1b, remain poorly defined. In this manuscript, the authors used CRISPR/Cas9-generated DCP1a/b knockout cells to begin to unravel some of the differential roles for human DCP1a and DCP1b in mRNA decapping, gene regulation, and cellular metabolism. While this manuscript presents some new and interesting observations on human DCP1 (e.g. human DCP1a/b KO cells are viable and can be used to investigate DCP1 function; only the EVH1 domain, and not its disordered C-terminal region which recruits many decapping cofactors, is apparently required for efficient decapping in cells; DCP1a and b target different subsets of mRNAs for decay and may regulate different aspects of metabolism), there is one key claim about the role of DCP1 in regulating DCP2-mediated decapping that is still incompletely or inconsistently supported by the presented data in this revised version of the manuscript.

Strengths & well-supported claims:

• Through in vivo tethering assays in CRISPR/Cas9-generated DCP1a/b knockout cells, the authors show that DCP1 depletion leads to significant defects in decapping and the accumulation of capped, deadenylated mRNA decay intermediates.<br /> • DCP1 truncation experiments reveal that only the EVH1 domain of DCP1 is necessary to rescue decapping defects in DCP1a/b KO cells.<br /> • RNA and protein immunoprecipitation experiments suggest that DCP1 acts as a scaffold to help recruit multiple decapping cofactors to the decapping complex (e.g. EDC3, DDX6, PATL1 PNRC1, and PNRC2), but that none of these cofactors are essential for DCP2-mediated decapping in cells.<br /> • The authors investigated the differential roles of DCP1a and DCP1b in gene regulation through transcriptomic and metabolomic analysis and found that these DCP1 paralogs target different mRNA transcripts for decapping and have different roles in cellular metabolism and their apparent links to human cancers. (Although I will note that I can't comment on the experimental details and/or rigor of the transcriptomic and metabolomic analyses, as these are outside my expertise.)

Weaknesses & incompletely supported claims:

(1) One of the key mechanistic claims of the paper is that "DCP1a can regulate DCP2's cellular decapping activity by enhancing DCP2's affinity to RNA, in addition to bridging the interactions of DCP2 with other decapping factors. This represents a pivotal molecular mechanism by which DCP1a exerts its regulatory control over the mRNA decapping process." Similar versions of this claim are repeated in the abstract and discussion sections. However, this claim appears to be at odds with the observations that: (a) in vitro decapping assays with immunoprecipitated DCP2 show that DCP1 knockout does not significantly affect the enzymatic activity of DCP2 (Fig 2C&D; I note that there may be a very small change in DCP2 activity shown in panel D, but this may be due to slightly different amounts of immunoprecipitated DCP2 used in the assay); and (b) the authors show only weak changes in relative RNA levels immunoprecipitated by DCP2 with versus without DCP1 (~2-3 fold change in Fig 3H, where expression of the EVH1 domain, previously shown in this manuscript to fully rescue the DCP1 KO decapping defects in cells, looks to be almost within error of the control in terms of increasing RNA binding). If DCP1 pivotally regulates decapping activity by enhancing RNA binding to DCP2, why is no difference in in vitro decapping activity observed in the absence of DCP1, and very little change observed in the amounts of RNA immunoprecipitated by DCP2 with the addition of the DCP1 EVH1 domain?

In the revised manuscript and in their response to initial reviews, the authors rightly point out that in vivo effects may not always be fully reflected by or recapitulated in in vitro experiments due to the lack of cellular cofactors and simpler environment for the in vitro experiment, as compared to the complex environment in the cell. I fully agree with this of course! And further completely agree with the authors that this highlights the critical importance of in cell experiments to investigate biological functions and mechanisms! However, because the in vitro kinetic and IP/binding data both suggest that the DCP1 EVH1 domain has minimal to no effects on RNA decapping or binding affinity, while the in cell data suggest the EVH1 domain alone is sufficient to rescue large decapping defects in DCP1a/b KO cells (and that all the decapping cofactors tested were dispensable for this), I would argue there is insufficient evidence here to make a claim that (maybe weakly) enhanced RNA binding induced by DCP1 is what is regulating the cellular decapping activity. Maybe there are as-yet-untested cellular cofactors that bind to the EVH1 domain of DCP1 that change either RNA recruitment or the kinetics of RNA decapping in cells; we can't really tell from the presented data so far. Furthermore, even if it is the case that the EVH1 domain modestly enhances RNA binding to DCP2, the authors haven't shown that this effect is what actually regulates the large change in DCP2 activity upon DCP1 KO observed in the cell.

Overall, while I absolutely appreciate that there are many possible reasons for the differences observed in the in vitro versus in cell RNA decapping and binding assays, because this discrepancy between those data exists, it seems difficult to draw any clear conclusions about the actual mechanisms by which DCP1 helps regulate RNA decapping by DCP2. For example, in the cell it could be that DCP1 enhances RNA binding, or recruits unidentified cofactors that themselves enhance RNA binding, or that DCP1 allosterically enhances DCP2-mediated decapping kinetics, or a combination of these, etc; my point is that without in vitro data that clearly support one of those mechanisms and links this mechanism back to cellular DCP2 decapping activity (for example, in cell data that show EVH1 mutants that impair RNA binding fail to rescue DCP1 KO decapping defects), it's difficult to attribute the observed in cell effects of DCP1a/b KO and rescue by the EVH1 domain directly to enhancement of RNA binding (precisely because, as the authors describe, the decapping process and regulation may be very complex in the cell!).

This contradiction between the in vitro and in-cell decapping data undercuts one of the main mechanistic takeaways from the first half of the paper; I still think this conclusion is overstated in the revised manuscript.

Additional minor comment:

• Related to point (1) above, the kinetic analysis presented in Fig 2C shows that the large majority of transcript is mostly decapped at the first 5 minute timepoint; it may be that DCP2-mediated decapping activity is actually different in vitro with or without DCP1, but that this is being missed because the reaction is basically done in less than 5 minutes under the conditions being assayed (i.e. these are basically endpoint assays under these conditions). It may be that if kinetics were done under conditions to slow down the reaction somewhat (e.g. lower Dcp2 concentration, lower temperatures), so that more of the kinetic behavior is captured, the apparent discrepancy between in vitro and in-cell data would be much less. Indeed, previous studies have shown that in yeast, Dcp1 strongly activates the catalytic step (kcat) of decapping by ~10-fold, and reduces the KM by only ~2 fold (Floor et al, NSMB 2010). It might be beneficial to use purified proteins here, if possible, to better control reaction conditions.

In their response to initial reviews, the authors comment that they tried to purify human DCP2 from E coli, but were unable to obtain active enzyme in this way. Fair enough! I will only comment that just varying the relative concentration of immunoprecipitated DCP2 would likely be enough to slow down the reaction and see if activity differences are seen in different kinetic regimes, without the need to obtain fully purified / recombinant Dcp2.

But it's not the 'same drug,' the author just said it's a more pure version that hasn't been tampered with.

Here at the very end, the author slips in an explicit pitch for market and corporate "freedom" that was long the central message of modern conservatism.

This language also seems to contradict earlier messages about "woke" corporate messages being inherently bad.

Welcome back.

In this lesson, I want to introduce the term "service models", specifically "cloud service models".

If you've ever heard or seen the term something as a service or x-a-a-s, then this is generally a cloud service model, and that's what I want to cover in this lesson.

Before I start, there are a few terms I'd like to introduce, which will make the rest of this lesson make more sense as well as being helpful throughout the course.

If you already know these, then that's fine.

It will just be a refresher.

But if these are new, then it's important to make sure you understand all these concepts because they're things that underpin a lot of what makes the cloud special.

Now, when you deploy an application anywhere, it uses what's known as an infrastructure stack.

An infrastructure stack is a collection of things, which that application needs, all stack on to each other.

Starting at the bottom, everything runs inside a facility, which is a building with power, with aircon, with physical security.

Everything uses infrastructure, so storage and networking.

An application generally requires one or more physical service.

These servers run virtualization, which allows them to be carved up into virtual machines.

These virtual machines run operating systems.

They could potentially run containers.

An example of this is Docker.

Don't worry if you don't know what these are, I'll be covering them later in the course.

Every application is written in a language such as Python, JavaScript, C, C++, C#, and all of these have an environment that they need to run in.

This is called a runtime environment.

An application needs data to work on, which it creates or consumes.

And then at the very top is the application itself.

Now, all of this together is an infrastructure stack or an application stack.

If you use Netflix or Office 365 or Slack or Google or your online bank or this very training site, it has parts in each of these tiers.

Even the application that you're using right now to watch this training is running on an operating system, which is running itself on a laptop, a PC or a tablet, which is just hardware.

The hardware uses infrastructure, your internet connection, and this runs in facilities, so your house or a coffee shop.

With any implementation of this stack, there are parts of the stack that you manage and there are parts of the stack which are managed by the vendor.

So if you're working in a coffee shop, they'll have specific people to manage the building and the internet connection, and that's probably not you.

But it's more likely that you are responsible for your laptop and the operating system running on top of it.

Now, this is true for any system.

Some parts you manage, some parts some people else manages.

You don't manage any part of Netflix, for example.

Netflix is an entity, manage everything end to end.

But if you do work in IT, then maybe you do manage all of the IT infrastructure stack or some parts of it.

The last term that I want to introduce is what's known as the unit of consumption.

It's what you pay for and it's what you consume.

It's the part of the system where from that point upwards in the infrastructure stack, you are responsible for management.

For example, if you procure a virtual server, then your unit of consumption is the virtual machine.

A virtual machine is just an operating system and an allocation of resources.

So your unit of consumption is the operating system.

In AWS, if you create a virtual machine known as an instance, then you consume the operating system.

If you use Netflix, though, then you consume the service and that's it.

You have no involvement in anything else.

The unit of consumption is what makes each service model different.

So let's take a look.

With an on-premise system, so that's one which is running in a building that your business owns, your business has to buy all parts of the stack.

It has to manage them all, pay for the upkeep and running costs of all of them.

And it has to manage the staff costs and risks associated with every single part of that stack.

Now, because it owns and controls everything, while it's expensive and it does carry some risks, it's also very flexible.

In theory, you can create systems which are tailor-made for your business.

Now, before cloud computing became as popular as it is now, it was possible to use something called data center hosting.

Now, this is similar to on-premises architectures, but when you use data center hosting, you place your equipment inside a building which is owned and managed by a vendor.

This meant that the facilities were owned and controlled by that vendor.

You as a business consumed space in that facility.

Your unit of consumption was a rack space.

If you rented three racks from a data center provider, they provided the building, the security, the power, the air conditioning and the staffing to ensure the service you paid for was provided.

All of the service models we use today are just evolutions of this type of model where more and more parts of the stack are handed off to a vendor.

Now, the cost change, the risks that are involved change and the amount of flexibility you have changed, but it's all the same infrastructure stack just with different parts being controlled by different entities.

So let's look at this further.

The first cloud service model that I want to talk about is infrastructure as a service or IaaS.

With this model, the provider manages the facilities, the storage and networking, the physical server and the virtualization and you consume the operating system.

Remember, a virtual machine is just an operating system with a certain amount of resources assigned.

It means that you still have to manage the operating system and anything above the operating system so any containers, the runtime, the data and your applications.

So why use IaaS as a service model?

With IaaS, you generally pay per second, per minute per hour fee for the virtual machine.

You pay that fee when you use that virtual machine and you don't pay when you don't use it.

The costs associated with managing a building, procuring and maintaining infrastructure and hardware and installing and maintaining a virtualization layer are huge and they're all managed by the vendor.

The vendor needs to purchase things in advance, pay licenses, pay staff to keep things running and manage the risks of data loss, hardware failure and a wealth of other things.

Using IaaS means that you can ignore all of those and let the vendor manage them.

IaaS is one of the most popular cloud service models.

Now, you do lose a little bit of flexibility because you can only consume the virtual machine sizes and capabilities that the provider allows, but there is a substantial cost reduction because of that.

In AWS, a product called Elastic Compute Cloud or EC2 uses the IaaS service model.

So in summary, IaaS is a great compromise.

You do lose a little bit in terms of flexibility, but there are substantial costs and risk reductions.

Okay, so let's move on.

Another popular service model is Platform as a Service or Pass.

Now, this service model is aimed more at developers who have an application they just want to run and not worry about any of the infrastructure.

With Pass, your unit of consumption is the runtime of the runtime environment.

So if you run a Python application, you pay for a Python runtime environment.

You give the vendor some data and your application and you put it inside this runtime environment and that's it.

You manage your application and its data and you consume the runtime environment, which effectively means that the provider manages everything else, containers, operating system, virtualization, service, infrastructure and facilities.

Now, let's review one final service model before we finish this lesson.

The final service model is Software as a Service or SaaS.

And with SaaS, you consume the application.

You have no exposure to anything else.

You pay a monthly fee for consuming the application.

You get it as a service.

Now, examples of SaaS products include Netflix, Dropbox, Office 365, Flickr, even Google Mail.

Businesses consume SaaS products because they are standard known services.

Email is email.

One email service is much like another.

And so a business can save significant infrastructure costs by consuming their email service as a SaaS solution.

They don't have much control of exactly how the email services can be configured, but there are almost no risks or additional costs associated with procuring a SaaS service.

IaaS, SaaS and SaaS are examples of cloud service models.

Now, there are others such as Function as a Service, known as SaaS, Container as a Service, Database as a Service or DBAAS, and there are many more.

For this lesson, the important points to understand are that the infrastructure stack exists in every service and application that you use.

The part of the stack is managed by you.

The part of the stack is managed by the provider.

And for every model, there is part of the stack which you consume, your unit of consumption.

That's the part that you pay for and generally the part that delineates between where the vendor manages and where you manage.

Now, again, I know this has been a fairly theory heavy lesson, but I promise you it will be invaluable as you go through the course.

Thanks for listening.

Go ahead, complete this video.

And when you're ready, join me in the next.

In this lesson, I want to cover theoretical topic which is really important to me personally, and something that I think is really valuable to understand.

That is, what is multi- and hybrid Cloud, and how do they relate to private and public Cloud platforms?

Now, why this matters is because AWS Azure and the Google Cloud Platform, they're all offering private Cloud environments which can be used in conjunction with their public Clouds.

So to be able to pick when and where to use them effectively, you need to understand when something is multi-Cloud and when it's hybrid Cloud because these are very different things.

So let's jump in and get started.

In the previous lesson, I covered the formal definition of Cloud computing.

Now, I know this was a very dry theoretical lesson, but hopefully you've come out of that understanding what a Cloud environment is.

Now, public Cloud, simply put, is a Cloud environment that's available to the public.

Many vendors are currently offering public Cloud platforms including AWS, Microsoft Azure, and Google Cloud.

These are all examples of public Cloud platforms.

They're public Cloud because they meet the five essential characteristics of Cloud computing and that they're available to the general public.

So to be public Cloud, it needs to first classify as a Cloud environment and then it needs to be available to the general public.

Now, you can if you have very specific needs or if you want to implement something which is highly available, you can choose to use multiple public Cloud platforms in a single system.

Now, that's known as multi-Cloud.

So multi-Cloud is using multiple Cloud environments and the way that you implement this can impact how successful it is.

Now, keeping things simple, you could choose to implement a simple mirrored system.

One part of your system could be hosted inside AWS and the other in Azure.

This means that you've got Cloud provider level resilience.

If one of these vendors fails, you'll know that at least part of your system will remain fully functional and running in the other.

Now, with regards to multi-Cloud, I would personally stay away from any products or vendors who attempt to provide a so-called single management window or single pane of glass if you want to use the jargon when using multiple Cloud platforms.

It is possible to manage multiple Cloud platforms as one single environment, but while it is possible, it abstracts away from these individual environments, relying on the lowest common feature set.

And so you do lose a lot of what makes each vendor special and unique.

So in this example, I could pick AWS and Azure.

I could abstract away from that using a third-party tool.

And when I wanted to provision a virtual machine, that tool would select which Cloud vendor to use.

The problem with that is that it would have to assume a feature set which is available in both of them.

So if AWS had any features that weren't available in Azure or vice versa, this third-party tool could not utilize them while staying abstracted away.

So that's a really important thing to understand.

Generally, when I'm thinking about multi-Cloud environments, I'm looking at it from a highly available perspective.

So putting part of my infrastructure in one and part in another.

It's much simpler and generally much more effective.

Now, each of these three Cloud vendors also offers a solution which can be dedicated to your business and run from your business premises.

This is a so-called private Cloud.

Now, for AWS, this is called AWS Outposts.

For Azure, it's the Azure Stack.

And for Google, it's Anthos.

Now, I want to make a very special point of highlighting that there is a massive difference between having on-premises infrastructure, such as VMware, Hyper-V, or Zen Server, versus having a private Cloud.

A private Cloud still needs to meet the five essential characteristics of Cloud computing, which most traditional infrastructure platforms don't.

So private Cloud is Cloud computing, which meets these five characteristics, but which is dedicated to you as a business.

So with VMware, Hyper-V, or Zen Server implementation, they're not necessarily private Cloud.

A lot of these platforms do have private Cloud-like features, but in general, the only environments that I consider true private Cloud are Outposts, the Azure Stack, and Google Anthos.

Now, it is possible to use private Cloud in conjunction with public Cloud.

And this is called hybrid Cloud.

It's hybrid Cloud only if you use a private Cloud and a public Cloud, cooperating together as a single environment.

It's not hybrid Cloud if you just utilize a public environment such as AWS together with your on-premises equipment.

Now, to add confusion, you might hear people use the term hybrid environment.

And in my experience, people use hybrid environment to refer to the idea of public Cloud used together with existing on-premises infrastructure.

So I'm going to try throughout this course to have separate definitions.

If I use the terms hybrid environment or hybrid networking, then that's different.

That simply means connecting a public Cloud environment through to your on-premises or data-center-based traditional infrastructure.

So there's a difference between hybrid Cloud, which is a formal definition, and then hybrid environment or hybrid networking.

So try and separate those and understand what's meant by age.

With true hybrid Cloud, you get to use the same tooling, the same interfaces, the same processes to interact with both the public and private components.

So let's summarize this.

Public Cloud means to use a single public Cloud environment such as AWS, Azure, or Google Cloud.

Private Cloud is to use on-premises Cloud.

Now, this is important.

This is one of the most important distinctions to make.

For it to be private Cloud, you need to be using an on-premises real Cloud product.

It needs to meet those five essential characteristics.

Multi Cloud means using more than one public Cloud.

So an example of this might be AWS and Azure, or AWS, Azure and Google.

They're examples of a multi Cloud deployment.

So using multiple public Clouds in one deployment, that's a multi Cloud environment.

And I mentioned that earlier in the lesson, that can be as simple as deploying half of your infrastructure to one public Cloud and half to the other, or using a third party tool that abstracts away from a management perspective.

But I would not recommend any abstraction or any third party tools.

Generally, in my experience, the best multi Cloud environments are those which use part of your infrastructure in one Cloud environment and part in the other.

Hybrid Cloud means utilizing public and private Clouds, generally from the same vendor, together as one unified platform.

And then lastly, and probably personally one of the most important points to me, Hybrid Cloud is not utilizing a public Cloud like AWS and connecting it to your legacy on-premises environment.

That is a hybrid environment or a hybrid network.

Hybrid Cloud is a very specific thing.

And I'm stressing this because it is important now to be an effective solutions architect.

You need to have a really good distinction between what public Cloud, private Cloud, multi Cloud, hybrid Cloud and hybrid environments are.

Understand all of those separate definitions.

Now that's all I wanted to cover in this lesson.

I hope it wasn't too dry.

I really do want to make sure that you understand all of these terms on a really foundational level because I think they're really important to be an effective solutions architect.

So go ahead, complete this lesson and then when you're ready, I'll see you in the next.

Something I need to keep in mind is that it's okay for vulnerability to be hard, to be tiring. I don't always have to share everything with everyone.

This is an interesting perspective that I think is missing from the way land recognitions are employed in progressive places. There is no 'why' of it, so it means nothing to most people. Also important that it's based in materialism. There is an actual history to trace back, not just liberal niceties.

In this lesson, I want to introduce Cloud Computing.

It's a phrase that you've most likely heard, and it's a term you probably think you understand pretty well.

Cloud Computing is overused, but unlike most technical jargon, Cloud Computing actually has a formal definition, a set of five characteristics that a system needs to have to be considered cloud, and that's what I want to talk about over the next few minutes in this lesson.

Understanding what makes Cloud 4 Cloud can help you understand what makes Cloud special and help you design cloud solutions.

So let's jump in and get started.

Now because the term Cloud is overused, if you ask 10 people what the term means, you'll likely get 10 different answers.

What's scary is that if those 10 individuals are technical people who work with Cloud day to day, often some of those answers will be wrong.

Because unlike you, these people haven't taken the time to fully understand the fundamentals of Cloud Computing.

To avoid ambiguity, I take my definition of Cloud from a document created by NIST, a NIST at the National Institute of Standards and Technology, which is part of the US Department of Commerce.

NIST creates standards documents, and one such document is named Special Publication 800-145, which I've linked in the lesson text.

The document defines the term Cloud.

It defines five things, five essential characteristics, which a system needs to meet in order to be cloud.

So AWS, Azure, and Google Cloud, they all need to meet all five of these characteristics at a minimum.

They might offer more, but these five are essential.

Now some of these characteristics are logical, and so they may surprise you.

So I've added a couple of things to the document, and I've added a couple of things to the document, and even though you and other business are probably sharing physical hardware, you would never know each other existed, and that's one of the benefits of Boolean.

But on to characteristic number four, which is rapid elasticity.

The NIST document defines this as capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly, outward, and inward, commensurate with demand.

To the consumer, the capabilities available for provisioning often appear to be unlimited, and can be appropriated in any quantity at any time.

Now I simplify this again into two points.

First, capabilities can be elastically provisioned and released to scale rapidly, outward and inward with demand, and in this case, capabilities are just resources.

And second, to the consumer, the capabilities available for provisioning often appear to be unlimited.

Now when most people think about scaling in terms of IT systems, they see a system increasing in size based on organic growth.

Elasticity is just an evolution of that.

A system can start off small, and when system load increases, the system size increases.

But, crucially, with elasticity, when system load decreases, the system can reduce in size.

It means that the cost of a system increases as demand increases, and the system scales, and decrease as demand drops.

Rapid elasticity is this process but automated, so the scaling can occur rapidly in real time with no human interaction.

Cloud vendors need to offer products and features, which monitor load, and allow automated provisioning and termination as load increases and decreases.

Now most businesses won't care about increased system costs.

If, for example, during sale periods, their profits increase, and because the system scales, along with that increased load and increased profits, the customers are kept happy.

Elasticity means that you don't have to, and indeed can't over provision, because over provisioning weighs money.

It also means that you can't under provision and experience performance issues for your customers.

It's how a company like Amazon.com or Netflix can easily handle holiday sales, or handle the load generated on the latest episode of Game of Thrones' release.

The second part is related to that.

A cloud environment shouldn't let you see capacity limits.

If you need 100 virtual machines or 1000, you should be able to get access to them immediately when required.

In the background, the provider is handling the capacity in a pooled way, but from your perspective, you should never really see any capacity limitations.

Now this is, in my opinion, the most important benefit of cloud, systems which scale in size in response to load.

So this is a really important one to make sure that a potential cloud environment offers in order to make sure that it is actually cloud.

Okay, let's move on to the final characteristic, and that's measured service.

Now this document defines this as cloud systems automatically control and optimize resource use by leveraging and metering capability at some level of abstraction appropriate to the type of service.

And it says that resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the service.

Now my simplified version of this is that resource usage can be monitored, controlled, reported, and built.

Traditional and non-cloud infrastructure work to using capex.

You pay for service and hardware in advance.

In the beginning, you had more capacity than you needed, so money was wasted.

Your demand grew over time, and eventually you purchased more service to cope with the demand.

If you did that too slowly, you had performance issues or failures.

With a true cloud environment, it offers on-demand billing.

Your usage is monitored on a constant basis.

You pay for that usage.

This might be a certain amount per second, a minute, an hour, or per day usage of a certain service, for example virtual machines.

Or it could be a certain cost for every gigabyte you store on a storage service for a given month.

You generally pay nothing in advance if it truly is a cloud platform.

If you consume a virtual server for a month, but then for 30 minutes at that month you use 100 virtual servers, then you should pay a small amount for the month and a much larger amount just for that 30 minutes.

Legacy vendors will generally want to feed or buy or lease a server.

If this is the case, they aren't cloud, and they probably don't support some of the massively flexible architectures the cloud allows you to build.

With that being said, that is everything I wanted to cover, so go ahead, complete this video, and when you're ready, I'll join you in the next.

Welcome back.

In a previous video I talked about YAML, which is a method of storing and passing data which is human readable.

In this video I want to cover JSON, which is the JavaScript object notation.

Let's jump in and take a quick look and note that many of the topics which I covered in the YAML video also apply here.

With conveying the same information, the format to do so is just different.

So unfortunately we have another mouthful of definition incoming.

So JSON, or the JavaScript object notation, is a lightweight data interchange format.

It's easy for humans to read and write and it's easy for machines to pass and generate.

That's what it says, but there are a few differences that you should be aware of before we move on.

JSON doesn't really care about indentation because everything is enclosed in something, so braces or brackets.

Because of this it can be much more forgiving regarding spacing and positioning.

And secondly because of that, JSON can appear initially harder to read.

But over time I've come to appreciate the way that JSON lays out the structure of its documents.

Now there are two main elements that you need to understand if you want to be competent with JSON.

First, an object or a JSON object.

And this is an unordered set of key value pairs enclosed by curly brackets.

Now from when you watched the YAML video, you should recognise this as a dictionary.

It's the same thing, but in JSON it's called an object.

The second main element in JSON is an array which is an ordered collection of values separated by commas and enclosed in square brackets.

Now from the YAML video you might recognise this as a list.

It's again the same thing, only in JSON it's called an array.

Now in both cases, arrays which are lists of values or objects which are collections of key value pairs, the value can be a string, an object, a number, an array, boolean, true or false, or finally null.

Now with these two high level constructs in mind, let's move on.

So this is an example of a simple JSON document.

Notice how even at the top level there are these curly brackets.

This shows that at the top level a JSON document is simply a JSON object, a collection of key value pairs separated by a colon.

In this example we have three keys.

We have cats, colours and finally, num of eyes.

And each key has a corresponding value in this example which is an array.

The top level key value pair has a value containing an array of cat names.

The middle has a value which is an array of the colour of the cats.

And then the last key value pair has a value which is a list of the number of eyes which each cat has.

Now JSON documents aren't limited to just arrays.

They can be much more complicated like this example.

Now this is a JSON document and every JSON document starts with a top level object, which is an unordered list of key value pairs surrounded by curly brackets.

This object has four key value pairs.

The keys are ruffle, truffles, penny and winky.

The value of each key is a JSON object, a collection of key value pairs.

So JSON objects can be nested within JSON objects, arrays can be ordered lists of JSON objects, which themselves can contain JSON objects.

And again, this lets you create complex structures which can be used by applications to pass or store data and configuration.

Now I'll admit it, I'm actually a fan of JSON.

I think it's actually easier to write and read than YAML.

Many people will disagree and that's fine.

With that being said though, that's everything that I wanted to cover in this video.

So go ahead and complete the video and when you're ready, I'll look forward to you joining me in the next.

Welcome to this video which will be a fairly high level introduction to YAML.

Now YAML stands for YAML 8 Markup Language and for any key observers that's a recursive acronym.

Now I want this video to be brief but I think it's important that you understand YAML's structure.

So let's jump in and get started.

YAML is a language which is human readable and designed for data serialization.

Now that's a mouthful but put simply it's a language for defining data or configuration which is designed to be human readable.

At a high level a YAML document is an unordered collection of key value pairs separated by a colon.

It's important that you understand this lack of order.

At this top level there is no requirement to order things in a certain way.

Although there may be conventions and standards none of that is imposed by YAML.

An example key value pair might be the key being cat1 and the value being raffle.

One of my cats in this example both the key and the value are just normal strings.

We could further populate our YAML file with a key of cat2 and a value of truffles and other cat of mine.

Or a key of cat3 and a value of penny and a key of cat4 and a value of winkey.

These are all strings.

Now YAML supports other types numbers such as one and two, floating point values such as 1.337, boolean so true or false and even null which represents nothing.

Now YAML also supports other types and one of those are lists known as arrays or other names depending on what if any programming languages that you're used to.

A list is essentially an ordered set of values and in YAML we can represent a list by having a key let's say Adrian's cats.

And then as a value we might have something that looks like this, a comma separated set of values inside swear brackets.

Now this is known as inline format where the list is placed where you expect the value to be after the key and the colon.

Now the same list can also be represented like this where you have the key and then a colon and then you go to a new line and each item in the list is represented by hyphen and then the value.

Now notice how for some of the values are actually enclosed in speech marks or quotation marks and so on.

This is optional.

All of these are valid.

Often though it's safe for you to enclose things as it allows you to be more precise and it avoids confusion.

Now in YAML indentation really matters.

Indentation is always done using spaces and the same level of indentation means that the things are within the same structure.

So we know that because all of these list items are indented by the same amount they're all part of the same list.

We know they're a list because of the hyphens.

So same indent always using hyphens means that they're all part of the same list, same structure.

Now these two styles are two methods for expressing the same thing.

A key called Adrian's cats whose value is a list.

This is the same structure.

It represents the same data.

Now there's one final thing which I want to cover with YAML and that's a dictionary.

A dictionary is just a data structure.

It's a collection of key value pairs which are unordered.

A YAML template has a top level dictionary.

It's a collection of key value pairs.

So let's look at an example.

Now this looks much more complicated but it's not if you just follow it through from the start.

So we start with a key value pair.

Adrian's cats at the top.

So the key is Adrian's cats and the value is a list.

And we can tell that it's a list because of the hyphens which are the same level of indentation.

But, and this is important, notice how for each list item we don't just have the hyphen and a value.

Instead we have the hyphen and for each one we have a collection of key value pairs.

So for the final list item at the bottom we have a dictionary containing a number of key value pairs.

The first has a key of name with a value of winky.

The second a key color with a value of white.

And then for this final list item a key, num of eyes and a value of one.

And each item in this list, each value is a dictionary.

A collection of one or more key value pairs.

So values can be strings, numbers, floats, booleans, lists or dictionaries or a combination of any of them.

Note how the color key value pair in the top list item, so the raffle dictionary at the top, its value is a list.

So this structure that's on screen now, we have Adrian's cats which are a value, has a list.

Each value in the list is a dictionary.

Each dictionary contains a name, key, with a value, a color key, with a value.

And then the third item in the list also has a num of eyes key and a value.

Now using YAML key value pairs, lists and dictionaries allows you to build complex data structures in a way which once you have practice is very human readable.

In this case, it's a database of somebody's cats.

Now YAML can be read into an application or written out by an application.

And YAML is commonly used for the storage and passing of configuration.

For now thanks for watching, go ahead, complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome to this video where I'm going to step through two concepts which I think should be mandatory knowledge for any solutions architect.

And for anyone else working in IT, these are also really useful.

First we have recovery point objective known as RPO and recovery time objective known as RTO.

Generally if you're a solutions architect helping a client, they will give you their required values for both of these.

In some cases you might need to work with key stakeholders within the business to determine appropriate values.

In either case if you get them wrong it can have a massive negative consequence to a business.

Let's jump in and get started.

I'm going to start by stepping through recovery point objective or RPO.

Recovery point objective or RPO is something that's generally expressed in minutes or hours.

And I'll illustrate this, let's say a given 24 hour period.

It starts on the left and midday, moves through midnight in the middle and finishes at 12 midday on the following day on the right.

I will need to consider an animal rescue business who have animals arriving to be fostered 24/7/365.

They have intake, vet exams and data restored within on-premises systems which need to be referred to constantly throughout the day.

At a certain point in time let's say 2am we have a server failure.

And for this example let's assume this is a single server which stores all the data for the organization and they have no redundancy.

Now this is a terrible situation but it's all too common for cash-strapped charities.

So remember, donate to your local animal rescue centre.

RPO is defined as the maximum amount of data and this is generally expressed in time that can be lost during a disaster recovery situation before that loss will exceed what the organization can tolerate.

If an organization tells you that they have an RPO of 6 hours, it means the organization cannot tolerate more than 6 hours of data loss when recovering from a disaster like this server failure.

Now different organizations will have different RPO values.

Banks logically will be able to tolerate although some know data loss because they deal with customer money.

Whereas an online store might be able to tolerate some data loss as they can in theory recreate orders in other ways.

Understanding how data can be lost during disaster recovery scenarios is key to understanding how to implement a given RPO requirement.

Let's consider this scenario that every 6 hours starting at 3pm on day 1 the business takes a full backup of the server which has failed.

So normally we have a backup at 3pm, one at 9pm, one at 3am and one at 9am.

So 4 backups every 24 hour period split by 6 hours.

In order to recover data from the failed server we need to restore a backup.

Ideally assuming that we have no failures it will be from the most recent backup.

Now successful backups are known as recovery points.

In the case of full backups each successful backup is one recovery point.

If you use full backups and incremental backups it's possible that to restore a single incremental backup, i.e. to use that one recovery point you'll need the most recent full backup and every incremental backup between that full and the most recent okay incremental backup.

So it's possible that a recovery point will need more than one backup.

With this scenario so the server failure at 2am the data loss will be the time between 2am and the most recent recovery point.

In this case 9pm at the previous day.

So this represents 5 hours of lost data.

If the failure occurred right after the 9pm backup had finished we'd have almost no data loss.

If the failure occurred one hour later at 3am we would have 6 hours of data loss.

Now the maximum loss of data for this type of scenario is the time between 2 successful backups.

In our case because backups occur every 6 hours then data loss could be a minimum of 0 if the server failure occurred right after the first backup finished or a maximum of 6 hours if the server failure occurred right before the next scheduled backup.

So when an organisation informs you that they have a requirement for an RPO of 6 hours they're telling you that they can only tolerate a maximum of 6 hours of data loss in a disaster scenario.

And as a general rule this means that you need to make sure that backups occur as often or more often than the RPO value provided by the organisation.

An RPO of 6 hours means at minimum a backup every 6 hours but to cope with random backup failure generally you'll want to make sure backups occur more frequently than required.

So in this example maybe once every 3 hours or maybe even once an hour.

Lower RPOs generally require more frequent backups which historically has resulted in higher cost for backup systems both in terms of media which also licensing, management overhead and other associated processes.

So RPO is a value which is generally given to you by an organisation or you might have to work with an organisation to identify an appropriate value and it states how much maximum loss of data in time the business can tolerate.

Different businesses will have different RPOs and sometimes even different RPOs for different systems within a single organisation.

A bank might have super low RPOs for its financial systems but it might tolerate a much higher one for its website.

If data changes less frequently the system is less important then higher RPO values are easier to tolerate for a business.

Now let's move on and cover recovery time objective or RTO.

To explain RTO we're going to use the same example of a 24 hour period starting at midday on one day moving through that day with midnight in the middle and then moving to midday the following day on the right.

For this example though I've moved the server failure to 10pm on day one and the most recent backup was 9pm so one hour before the failure.

As you know now this means assuming the backup is working i.e. it's a valid recovery point that the data loss will be one hour, the time between the 9pm backup and the 10pm server failure.

RTO or recovery time objective simply put is the maximum tolerable length of time that a system can be down after a failure or disaster occurs.

Now once again just as with RPO this value is something that a business will give you as a directive or alternatively it's something that you'll work with a business on to determine a suitable value.

Also just as with RPOs different businesses will have different RPOs.

A bank will have a much lower RPO for its banking systems than a cafe for its website and an organisation will generally have different RPOs for its different systems.

Critical systems will have lower RPOs and less important systems can potentially have higher RPOs.

Now looking at the RPO definition in a different way, if the animal rescue business had an RPO of 13 hours it would mean that for a server failure which occurred at 10pm the IT operations team would have as a maximum until 11am the following day to fully restore the system to an operational state.

Now something which is really important that I need you to understand and this isn't always obvious especially if you haven't worked on or with a service desk before.

Recovery time of the system begins at the moment of the failure that's when the clock starts ticking and it ends not when the issue is fixed but when the system is handed back to the business in a fully tested state.

So in this example the clock starts at 10pm and to meet an RPO of 13 hours the server needs to be running again fully working by 11am the following day.

So you might ask why I'm stressing this point?

Well RPO isn't just about technical things, the biggest impacts on RPOs are things which as a technical person you might not always identify.

I want to step through some considerations which might impact the ability of this animal rescue's operation team to meet this 13 hour RPO directive.

The first thing which might not be immediately obvious is that to recover a system you need to know that that system has failed.

If the failure occurs at 10pm while the recovery time starts at that point the ability to recover only really starts when you're made aware that the system has failed.

So how long till the operations team know that there is an issue?

Is there monitoring in place on this service?

Is it reliable because too many false positives will have people ignoring outage notifications?

How will the monitoring system notify staff?

Will it wake staff who are sleeping?

Will it be the correct staff, staff who are empowered to begin a recovery process?

This is the real starting point to any recovery process and it often adds lag onto the start of the process.

Even with major internet applications that I use it's not uncommon for outages to occur and then take a further 15 to 30 minutes before the vendor is actively aware and investigating a fault.

Now don't underestimate the importance of effective monitoring and notification systems.

Beyond that make sure that you've planned and configured these processes in advance.

Waking up a junior operations person who has no ability to make decisions or no ability to wake up senior staff members is useless in this scenario.

Best case this part of the process takes some time so make sure that it's built into your planning.

Now let's move on and assume that we do have somebody in the ops team who can begin the process.

Well step number two is going to be to investigate the issue.

It might be something which is fixable quickly or it might be that a server is literally on fire.

Somebody needs to take the time to make the final decision to perform a restore if required and again this will take some time.

Moving on if we assume that we are going to do a restore we need to focus on the backup system.

What type of backups do we have?

Some take longer to restore versus others.

If it's a tape backup system where are the tapes?

Where is the tape drive or the loader?

Who needs to restore it?

Do they need to be in a specific physical location?

How does the restore happen?

Is there a documented process?

And is the person or one of the people who can perform the restore available and awake?

All of these are critical in your ability to begin the restore process and they all take time.

Now this type of disaster recovery scenario if you don't have a documented and tested process this can also be really stressful.

And a stressful situation late at night without your team around you this is when mistakes happen.

But let's assume that we have a working backup system.

We know where the backup media is and we have that and somebody who can operate the restore.

The next step is where we're restoring to.

The server we had has had a major failure or might literally be on fire.

Do you think about this in advance?

These choices so what are we restoring on?

Do we have a spare?

Do we need to order another server?

Are we using physical or virtual servers?

Or are we even forced to use a secondary disaster recovery site because not only is the server on fire but also is the server room.

Many people miss these elements when thinking about RTO but these are the things which really matter.

A badly documented process to restore servers or a slow notification system might add additional hours.

But having to order new server hardware could add days to a recovery time.

And finally for those wells that the restore has completed the operations team thinks the service is back up and running.

Then there needs to be time allocated for business testing, user testing and final handover.

This isn't quick and has to be done before you consider recovery to be complete.

This entire process end to end is what recovery is.

And so if the business has a 13 hour RTO you need to make sure that all of this process in its entirety fits into that 13 hours.

So that's what RTO is, a value given to you by a business or something that you help a business identify.

It's the maximum tolerable time to recover a system in the event that a disaster occurs.

It's the end to end process so this includes fault identification, restoration and final testing and handover.

So it's really important that when you're planning the recovery process for a system and you're given an RTO value by the business you're sure that you have time to perform all of these individual steps.

Now let's quickly summarize what we've learned before we finish this video.

So RPO is how much data the maximum data expressed in time at a business can lose.

So this is amount of data expressed in time beyond which it's not tolerable to the business.

So worst case this is the time between successful backups.

In general to implement more and more demanding RPO directives you need more frequent backups.

This means more cost but it does result in a lower RPO.

So when you see RPO think maximum data loss.

RTO or recovery time objective is a directive from the business which is a maximum restore time that that business can tolerate.

And this is end to end from identification through to final testing and handover.

So this can be reduced by effective planning, monitoring, notification, formal processes, spare hardware, training and more efficient systems such as virtual machines or AWS.

So RTO is the maximum time from when a failure occurs through to when the business will need that system back up and running in an operational state.

And by thinking about this in advance you can make your recovery process more efficient and meet more demanding RTOs from the business.

Now different businesses and different systems within the businesses will have different RPO and RTO values.

Generally the more critical a system is the lower and thus more demanding the RPO and RTO values will be.

And non-critical systems of business is usually more willing to tolerate higher and so less demanding RPO and RTO values.

Because generally what you're looking for is a gold lock point where you're as close to the true business requirements as possible.

Now as a solutions architect it's often the case that the business isn't aware of appropriate RTO and RTO values.

And so one of the core duties when designing new system implementations is to work with the business and understand which systems are critical and which can tolerate more data loss or recovery outages.

And by appropriately designing systems to match the true business requirements you can deliver a system which meets those requirements in a cost effective way.

Now at this point that's everything I want to cover about RPO and RTO at a high level.

If you're doing one of my ADL West courses as you're going through the course consider how you think the products and services being discussed would affect the RPO and RTO's of systems designed utilizing those products.

And if appropriate I'll be discussing exactly how features of those products can influence RPO and RTO values.

At this point though that's everything I wanted to cover in this video.

Thanks for watching.

Go ahead and complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome back.

In this fundamentals video, I want to briefly talk about Kubernetes, which is an open source container orchestration system.

You use it to automate the deployment, scaling and management of containerized applications.

At a super high level, Kubernetes lets you run containers in a reliable and scalable way, making a vision fuse of resources, and lets you expose your containerized applications to the outside world or your business.

It's like Docker, only with robots automated and super intelligence for all of the thinking.

Now, Kubernetes is a cloud agnostic product, so you can use it on premises and within many public cloud platforms.

Now, I want to keep this video to a super high level architectural overview, but that's still a lot to cover.

So let's jump in and get started.

Let's quickly step through the architecture of the Kubernetes cluster.

A cluster in Kubernetes is a highly available cluster of compute resources, and these are organized to work as one unit.

The cluster starts with a cluster control plane, which is the part which manages the cluster.

It performs scheduling, application management, scaling and deployment, and much more.

Compute within a Kubernetes cluster is provided via nodes, and these are virtual or physical servers, which function as a worker within the cluster.

These are the things which actually run your containerized applications.

Running on each of the nodes is software, and at minimum, this is container D or another container runtime, which is the software used to handle your container operations.

And next, we have KubeLit, which is an agent to interact with the cluster control plane.

And on each of the nodes communicates with the cluster control plane using Kubernetes API.

Now, this is the top level functionality of the Kubernetes cluster.

The control plane orchestrates containerized applications which run on nodes.

But now let's explore the architecture of control planes and nodes in a little bit more detail.

On this diagram, I've zoomed in a little.

We have the control plane at the top and a single cluster node at the bottom, complete with the minimum Docker and KubeLit software running for control plane communications.

Now, on to step through the main components which might run within the control plane and on the cluster nodes.

Keep in mind, this is a fundamental level video.

It's not meant to be exhaustive.

Kubernetes is a complex topic, so I'm just covering the parts that you need to understand to get started.

Now, the cluster will also likely have many more nodes.

It's rare that you only have one node unless this is a testing environment.

Now, first, I want to talk about pods and pods at the smallest unit of computing within Kubernetes.

You can have pods which have multiple containers and provide shared storage and networking for those pods.

But it's very common to see a one-container, one-pod architecture, which as the name suggests, means each pod contains only one container.

Now, when you think about Kubernetes, don't think about containers.

Think about pods.

You're going to be working with pods and you're going to be managing pods.

The pods handle the containers within them.

Architecturally, you would generally only run multiple containers in a pod when those containers are tightly coupled and require close proximity and rely on each other in a very tightly coupled way.

Additionally, although you'll be exposed to pods, you'll rarely manage them directly.

Pods are non-permanent things.

In order to get the maximum value from Kubernetes, you need to view pods as temporary things which are created, do a job, and are then disposed of.

Pods can be deleted when finished, evicted for lack of resources, or the node itself fails.

They aren't permanent and aren't designed to be viewed as highly available entities.

There are other things linked to pods which provide more permanence, but more on that elsewhere.

So now let's talk about what runs on the control plane.

Firstly, I've already mentioned this one, the API, known formally as Q-API server.

This is the front end for the control plane.

It's what everything generally interacts with to communicate with the control plane, and it can be scaled horizontally for performance and to ensure high availability.

Next, we have ETCD, and this provides a highly available key value store.

So a simple database running within the cluster, which acts as the main backing store for data for the cluster.

Another important control plane component is Q-scheduler, and this is responsible for constantly checking for any pods within the cluster which you don't have a node assigned.

And then it assigns a node to that pod based on resource requirements, deadlines, affinity, or anti-affinity, data locality needs, and any other constraints.

Remember, nodes are the things which provide the raw compute and other resources to the cluster, and it's this component which makes sure the nodes get utilized effectively.

Next, we have an optional component, the Cloud Controller Manager, and this is what allows Kubernetes to integrate with any cloud providers.

It's common that Kubernetes runs on top of other cloud platforms such as AWS, Azure, or GCP, and it's this component which allows the control plane to closely interact with those platforms.

Now, it is entirely optional, and if you run a small Kubernetes deployment at home, you probably won't be using this component.

Now, lastly, in the control plane is the Q-Controller Manager, and this is actually a collection of processors.

We've got the node controller, which is responsible for monitoring and responding to any node outages, the job controller, which is responsible for running pods in order to execute jobs, the endpoint controller, which populates endpoints in the cluster, more on this in a second, but this is something that links services to pods.

Again, I'll be covering this very shortly.

And then the service account and token controller, which is responsible for account and API token creation.

Now, again, I haven't spoken about services or endpoints yet, just stick with me.

I will in a second.

Now, lastly, on every node is something called K-Proxy, known as Cube Proxy, and this runs on every node and coordinates networking with the cluster control plane.

It helps implement services and configs rules allowing communications with pods from inside or outside of the cluster.

You might have a Kubernetes cluster, but you're going to want some level of communication with the outside world, and that's what Cube Proxy provides.

Now, that's the architecture of the cluster and nodes in a little bit more detail, but I want to finish this introduction video with a few summary points of the terms that you're going to come across.

So, let's talk about the key components.

So, we start with the cluster, and conceptually, this is a deployment of Kubernetes.

It provides management orchestration, healing, and service access.

Within a cluster, we've got the nodes which provide the actual compute resources, and pods run on these nodes.

A pod is one or more containers, and it's the smallest admin unit within Kubernetes, and often, as I mentioned previously, you're going to see the one container, one pod architecture.

Simply put, it's cleaner.

Now, a pod is not a permanent thing, it's not long-lived.

The cluster can and does replace them as required.

Services provide an abstraction from pods, so the service is typically what you will understand as an application.

An application can be containerized across many pods, but the service is the consistent thing, the abstraction.

Service is what you interact with if you access a containerized application.

Now, we've also got a job, and a job is an ad hoc thing inside the cluster.

Think of it as the name suggests, as a job.

A job creates one or more pods, runs until it completes, retries if required, and then finishes.

Now, jobs might be used as back-end isolated pieces of work within a cluster.

Now, something new that I haven't covered yet, and that's Ingress.

Ingress is how something external to the cluster can access a service.

So, you have external users, they come into an Ingress, that's routed through the cluster to a service, the service points at one or more pods, which provides the actual application.

So, Ingress is something that you will have exposure to when you start working with Kubernetes.

And next is an Ingress controller, and that's a piece of software which actually arranges for the underlying hardware to allow Ingress.

For example, there is an AWS load balancer, Ingress controller, which uses application and network load balancers to allow the Ingress.

But there are also other controllers such as Nginx and others for various cloud platforms.

Now, finally, and this one is really important, generally it's best to architect things within Kubernetes to be stateless from a pod perspective.

Remember, pods are temporary.

If your application has any form of long-running state, then you need a way to store that state somewhere.

Now, state can be session data, but also data in the more traditional sense.

Any storage in Kubernetes by default is ephemeral, provided locally by a node, and thus, if a pod moves between nodes, then that storage is lost.

Conceptually, think of this like instant store volumes running on AWS EC2.

Now, you can configure persistent storage known as persistent volumes or PVs, and these are volumes whose lifecycle lives beyond any one single pod, which is using them.

And this is how you would provision normal long-running storage to your containerized applications.

Now, the details of this are a little bit beyond this introduction level video, but I wanted you to be aware of this functionality.

OK, so that's a high-level introduction to Kubernetes.

It's a pretty broad and complex product, but it's super powerful when you know how to use it.

This video only scratches the surface.

If you're watching this as part of my AWS courses, then I'm going to have follow-up videos which step through how AWS implements Kubernetes with their EKS service.

If you're taking any of the more technically deep AWS courses, then maybe other deep-dive videos into specific areas that you need to be aware of.

So there may be additional videos covering individual topics at a much deeper level.

If there are no additional videos, then don't worry, because that's everything that you need to be aware of.

Thanks for watching this video.

Go ahead and complete the video, and when you're ready, I look forward to you joining me in the next.

Welcome back in this video I want to talk about the DNS signing ceremony.

If you're imagining confetti and champagne right now, it's the opposite of that kind of ceremony.

This ceremony is all about controlling the keys to the internet, more specifically the trust anchor of the DNS system.

It's one of the most important meetings which occur in the technical space.

Pretty much everything that you use on the internet is enabled by the technical act which occurs within the ceremony.

Now before I cover what the ceremony is, we need to understand why anything like this is needed.

Trust within a DNS zone is normally provided via the parent zone of that zone.

The parent zone has a DS record which is a hash of the public key signing key of the child zone, and that's how the trust chain is created.

In the case of the root zone, there is no parent zone and this means there's nothing to provide that trust.

And so a more rigorous process is required, something which is secure enough that the output can be absolutely trusted by every DNSSEC resolver and client.

And we refer to this concept as a trust anchor.

Locked away within two secure locations, one in California and another in Virginia, is what amounts to the keys of the internet.

The private DNS root key signing key known as a KSK.

Now it's impossible to overstate how important this set of keys is to the internet.

They rarely change and the trust in them is hard coded into all DNSSEC clients.

With them you can define what's valid on the DNSSEC root zone.

Because of this, every child's top level domain, every child's zone, inside those and every DNS record, these are locked away, protected and never exposed.

And they use redundant hardware security modules also redundant across physical locations.

Now I'll detail this more in a second.

With access to the private keys is controlled via the fact that HSMs are used.

You can only use the HSMs in tightly controlled ways.

The keys never leave those HSMs, those HSMs never leave those locations.

And you can only use them within those locations if you have the right group of people.

And people can only get into those locations after going through a rigorous multi-stage ID process.

Now why this is important is because we all know a public part of this key.

It's part of the DNS key record set within the root zone along with the public zone signing key.

To recite this, every DNSSEC client and resolver on the planet explicitly trusts this key, this key signing key.

And if we have this public root zone key signing key, we can verify anything signed by the private key, the one that's locked away within the hardware security modules.

Because the security of the private root key signing key is so tight, it's not practical to use constantly.

And so there's another key pair which controls the security of the DNS root zone.

This is known as the root zone ZSK or zone signing key.

The whole function of this massively controlled ceremony is to take the root zone ZSK, take this into the ceremony, sign it with the private root zone KSK, within these hugely tightly controlled conditions and then produce as an output the root zone RRSIG DNS key.

This single record is why DNS from this level down through the top level domains and into the domain zones, this is why it's all trusted because the root zone is trusted via this signing process.

Now talking through the detail of the signing ceremony would take too long.

And so I've included a link attached to this video which gives a detailed overview of the ceremonial process.

These ceremonies are public and recorded and links detailing all of this process is included attached to this video.

What I want to do now is to talk in a little bit more detail about why this process is so secure.

The signing ceremony itself takes place, as I mentioned previously, in one of two secure locations.

Now there are a few key sets of people involved.

I'll not show them all on screen, but we have the ceremony administrator, an internal witness, the credential safe controller, the hardware safe controller, and then crypto officer 1, crypto officer 2 and crypto officer 3.

Now there are a total of 14 of these crypto officers.

Seven of them are affiliated with each of the locations and at least three are required to attend for the process to work.

Logistically, dates and times of it 24 to 5 are available to ensure some level of resilience.

The most important part of the whole process hardware wise is the hardware security module or HSM, which is the hardware which contains the root zone private key signing key.

This device is protected and can only be interacted with via the ceremony laptop which is connected to the HSM over ethernet and this is only operated by the ceremony administrator.

The laptop has no battery and no storage.

It's designed to be stateless and only used to perform the ceremony and not store any data afterwards.

Now the HSM device can only be used when crypto officers use their allocated cards.

What's being signed are the public key signing key and zone signing keys.

We'll actually a pack of them to allow for rotation between this ceremony date and the next.

The HSM device via the ceremony laptop then outputs the signatures for these keys which become the DNS key, RRC records for the DNS root zone.

Now again this process happens every three months.

It's generally broadcast, notes of the process are available and it's publicly audited.

Now this is a summary of the process.

The level of security procedure which goes into ensuring that groups of human participants can't collude and corrupt the process is extreme.

I've included additional links attached to this video which provide more detail if you're interested.

For this video I just want you to have an understanding of why the ceremony is so important.

So during the ceremony we take the root key signing keys which everything trusts but which are too important to be used day to day.

And we use those to sign root zone signing keys which can be used on a more operational basis and these can be used to sign individual resource record sets within the root zone.

And it's that public and private zone signing key pair which is then used to create the chain of trust which allows trust in top level domains.

And then those top level domains can pass that trust on to domains and then in domains that trust can be passed to individual resource record sets.

And this all has to happen because we have nothing above the root key signing keys in the DNS.

There are trust anchor.

Nothing makes us trust them other than the trust itself.

And the ceremony ensures that it's almost impossible to corrupt that trust.

At this point that is everything I wanted to cover in this video.

So go ahead and complete the video and when you're ready I look forward to you joining me in the next.

Welcome back.

In this video, I want to step through how DNSSEC works inside his own, specifically how it allows a DNSSEC resolver or client to validate any resource records within his own.

This video is focusing on the data integrity part of DNSSEC.

And coming up after this is another video where I'll cover the chain of trust and origin authentication benefits which DNSSEC provides in a lot more detail.

Now, because this video covers digital signing within DNSSEC, it's important that you've watched my previous videos on DNS, on hashing, and on digital signing.

If you haven't, all of those videos will be linked in the description.

If you have, then let's jump in and get started.

Now, to understand DNSSEC, I first need to introduce a term, and that's a resource record set or RR set.

Let's look visually at what this is.

We'll start with the DNS zone of ican.org, and I'm using this as an example in this lesson as it's one which I know to be DNSSEC enabled.

Now, inside his zone, we have a number of resource records.

First, www.ican.org, which is a CNAME record.

And remember, CNAMES point at other records.

And in this case, it points to two other records.

One of them is an A record, so IP version 4, and the other is an AAA record, which is IP version 6.

Now, finally, we have four MX records for the domain pointing at four different mail exchange servers.

Now, each of these are resource records.

I'm showing a total of seven.

So what's a resource record set?

Well, a resource record set or RR set is any records of the same name and the same type.

So in the case of the left three, this means each of them is their own RR set.

But in the case of the MX records, they all have the same name, so ican.org, and they're all MX records.

And this means that all four of these are inside one RR set.

RR sets are just sets of resource records.

They make it easier to deal with records in groups versus single records.

And in the case of DNS set, it keeps things manageable in other ways, but more on that in a second.

Now, this is what an RR set might look like if you actually interact with DNS.

Notice how all the names are the same and all the types are the same.

So why do you need to know this?

Well, because RR sets are used within DNS set.

Right now, there's no way to tell if any of these resource records are valid.

DNS set provides this functionality, but it's not individual resource records which are validated by DNS set.

It's resource record sets or RR sets.

Now, let's take a look at how this works.

So DNS allows us to validate data integrity of record sets within DNS.

It doesn't work on individual records, rather it works on sets of records, RR sets.

Let's take a look at how.

So we start with the same ican.org zone, and inside here I'm going to step through one RR set example, the set of four resource records which make up the MX RR set.

Right now, without DNS set, if a bad actor found a way to change or make you think that these records have changed, then email delivery could, in theory, be redirected.

DNS set helps prevent that using two features.

First, we have RR sync, which stores a digital signature of an RR set using public and private pairs of keys.

This one key pair is known as the zone signing key or ZSK.

The private part of this key pair is sensitive and it's not actually stored within the zone.

It's kept offline, it's kept separated, but you need to know that it exists.

Like any private key, you need to keep this key safe and not accessible from the public domain.

So once again, an RR sync contains a digital signature of an RR set.

So we take the RR set which is plain text, we run it through a signing process, let's call this the digital signature atron 9000.

In reality, it's just a standard cryptographic process.

This process uses the private part of the ZSK to create a signature, and this is why it's important to keep the private part of this key safe.

This output, the signature can be stored alongside the plain text RR set in the zone using the same name, but the record type is RR sync.

Any normal DNS clients will only see the RR set, any DNS set clients will see the RR set and the corresponding RR sync.

Now this uses digital signing and hashing.

If the RR set changes, the RR sync has to be regenerated in order to be valid.

If the RR set changes without a corresponding change to the RR sync, the result is an invalid signature.

And so you can tell if anything has changed without the approval of the person controlling the private zone signing key.

And this is because only the private part of the zone signing key can be used to sign RR sets creating an RR sync.

Assuming that you trust that the private zone signing key is in safe hands, then you know that if there's a valid RR sync for a corresponding RR set, that RR set is in a valid state created by the zone admin, and if it changes, you can tell.

Now the important question is, how can a DNS client or resolver verify the RR sync?

For that, there's another piece to the DNS set puzzle.

We need the public part of the zone signing key to be able to verify signatures or RRs sync created using the private part.

Lucky for us, public parts of the key pairs aren't sensitive and don't need to be guarded.

We just need a way to make them available.

So consider this scenario, we have the same ICANN.org domain.

We also have a DNS set resolver here at the bottom.

How do we know it's a DNS set resolver?

You'll just have to trust me, but it is a super smart resolver.

Now inside the zone, we have the MX RR set for the ICANN.org zone.

We also have the MX RR sync, which remember is a signature for the RR set created using the private part of the zone signing key.

Inside a DNS set enabled zone, you'll find another record, the DNS key record.

The DNS key record stores public keys.

These public keys can be used to verify any RRs in the zone.

DNS key records can store the public key for the zone signing key, so the ZSK, and also a different type of key, the key signing key or KSK.

But more on this in a second.

We're going to do this step by step.

This is what a DNS key record might look like, and because it can store different public keys, there's a flag value.

A value of 256 means that it's a zone signing key, and a value of 257 means it's a key signing key.

So the top one here, this is the zone signing key, and it's this value, which is the last piece of the puzzle.

It means the DNS resolver can take the RR set, which remember is the plaintext part, and using this together with the matching RRsig and the DNS key record, it can verify that both the RRsig matches the RR set, and the signature was generated using the private part of the zone signing key.

And the result of this is that our DNS-seq-capable resolver at the bottom can verify the RR set is valid and hasn't been compromised.

Now this is all assuming, and this is a big assumption, that we trust the DNS key, specifically the zone signing key.

You have to trust that only the zone admin has the private part of the key, and you also have to trust that it's the correct zone signing key.

If you do, you can trust the RRsig and matching RR set are valid.

Now a real-world comparison of this would be to imagine if somebody shows you an ID card which has their photo on it.

The photo ID only proves their identity if you trust the photo ID is real and it was created by a genuine authority entity.

In humans, this trust is a bit fake, that's why fake IDs are such a problem.

This isn't a problem with DNS-seq, because as you'll see, we have a chain of verifiable trust all the way to the DNS route.

The DNS key record also requires a signature, and this means a matching RRsig record to validate that it hasn't been changed.

The DNS key record, though, is assigned with a different key, the key signing key, or KSK, so as the name suggests, this key isn't used for signing anything in the zone, instead it's used for signing keys.

So the zone signing key is used for signing everything in a zone, so to create most RRsig records, except the DNS key records, these are signed by the key signing key creating the DNS key RRsig record.

Now I get it, I've just introduced another type of key, so let's look at how this all fits together within a DNS zone.

At this point, I want to cover two really important points about DNS-seq.

First, how does a key signing key fit into all this and why do we have them?

And second, what mechanism allows us to trust a zone?

We know that RRsig records let DNS-seq resolvers verify record sets, but how do we know the keys used within a zone themselves can be trusted?

To illustrate both of these, let's start with the same ICANN.org zone, and then off to the right side, a related area, but containing more sensitive things, this might be a physical key store like a FileSafe or a HSM device.

The container here to start with is the private zone signing key, and this is used together with an RRset record to create a corresponding RRsig record.

Then, the public part of this zone signing key is stored in the DNS key record.

The flag of 256 tells us that it's a zone signing key.

At this point, I want to pause and take a quick detour.

If this was all that we had, so the DNS key, we couldn't trust it.

Somebody could swap out the DNS key record, put in a new public key in there, use the private part of that fake key to regenerate a fake RRsig, adjust the RRset and take over email for this domain.

We need a way of ensuring the zone signing key is trusted.

If we didn't have some kind of trust chain, we would need to manually trust every zone within DNS.

That would defeat the purpose of having a globally distributed system.

So, the way that this works is that this zone, so ICANN.org, is linked cryptographically to the parent zone, which is .org.

So, just like with normal DNS where name server records are used in the org zone to delegate to domains such as ICANN.org, the org parent zone also has a way to explicitly state that we can trust ICANN.org as a zone.

And I'll talk about exactly how this works in the next video.

For now, I want to focus on this zone.

Now, if we use a single key, so just the zone signing key, that would work.

But this would mean if we ever wanted to change the zone signing key, then we would have to involve the .org parent zone.

Best practice is that we want to be cycling keys fairly often, and doing that where it also requires updates up the chain would become inefficient.

And so we have this new key pair, the key signing key, or KSK.

Now, there's a private part in a public heart.

The private part is used to sign DNS key record sets, which creates an RRSIG of the DNS key.

And this makes it easy to change the zone signing key used for a zone.

We just have to regenerate all of the RRSIG records, update the DNS key record set, and then regenerate the RRSIG of the DNS key record set using the private key signing key.

All of this is inside our zone only.

It doesn't involve the parent zone in any way.

We store the public part of the key signing key in the DNS key record set.

But now we have a new problem.

How can we trust the key signing key?

Well, spoiler, it's referenced from the parent zone, in this case .org.

So remember how I said that the DNS key record set stored both zone signing and key signing public keys?

Well, this is the main point of trust for the zone.

This is how trust is conveyed into the zone.

Because the parent zone links to the public key signing key of our zone, assuming we can trust the .org parent zone, because it references our zone's key signing key, we can trust our zone's key signing key.

This key signing key signs our zone signing key, and the zone signing key signs all of the RR sets to create RR6.

We have a chain of trust created between two different layers of DNS, specifically DNSSEC.

Now, we're at the point now where you should understand how DNSSEC validates things within a single zone.

How it uses RR6 to validate RR sets, how it uses the DNS key records to get the public keys to deal with that validation using the zone signing key.

How a key signing key is used to create an RRSIG at the DNS key, which allows the validation of that DNS key record.

And how the parent domain or parent zone trusts the public key signing key at the child domain or zone.

Now, you don't know how this trust occurs yet.

That's something I'm going to be talking about in the next video.

I've also stepped through why two different keys are needed.

Using a zone signing key for signing within a zone, and a key signing key for signing that key, that allows an admin split.

The key signing key can be referenced from the parent zone, while the zone signing key is used exclusively within the zone.

And this means that a zone signing key can be changed without requiring any changes to the parent zone.

So what position does that put us in?

With this functionality using DNSSEC, what have we gained in the way of additional security?

Well, we can now verify the integrity of data within this specific DNS zone.

So, we've eliminated the DNS cache poisoning example.

Assuming we trust the key signing key, and also assuming the key signing key hasn't been changed as part of an exploit, then we can trust all of the information contained within a zone.

Now, in the next video, I'm going to step you through exactly how DNSSEC creates this chain of trust.

And this allows a parent zone to indicate that it trusts the key signing key used within a child zone.

And the same architecture at every level of DNSSEC means that we can create this entire end-to-end chain of trust, which can be verified cryptographically.

Now, at this point, that's all I wanted to cover in this video.

In the next video, I'm going to step through how this trust from a parent zone to a child zone is handled within the DNSSEC hierarchy.

And we'll go through how the query flow works step by step.

For now, though, go ahead and complete this video, and when you're ready, I look forward to you joining me in the next.

Welcome back and in this video of my DNS series I want to talk about DNSsec.

Now this is the first video of a set which will cover DNSsec which provides us with a lot of additional functionality and so requires a few dedicated videos.

Now to get the most from this video it's important that you've watched my previous videos on DNS as well as on hashing and digital signing.

If you haven't all the videos that you need I'll link in the description and you should watch those first or have equivalent knowledge.

DNSsec is a secure add-on for DNS, it provides additional functionality.

In this video I want to set the scene by talking about why we need DNSsec so let's jump in and get started.

Now I promise this bit will be super quick but I do need to use some bullet points.

I hate bullet points as much as anyone but sometimes they're just the quickest way to present information so let's go and please stick with me through this first set of important points.

Now DNSsec provides two main improvements over DNS.

First data origin authentication and this allows you to verify that the data that you receive is from the zone that you think it is.

Are the records returned to you from Netflix.com from the real Netflix.com.

If you're looking at cash results are they really from that original zone?

Second DNSsec provides data integrity protection so has the data that you receive been modified in any way since it was created by the administrator of the zone.

So if you have Netflix.com data is that the same un-changed Netflix.com data which the administrator of the Netflix.com zone created.

Now it does both of these things by establishing a chain of trust between the DNS route and DNS records but it does this in a cryptographically verifiable way where DNS has some major security holes DNSsec uses public key cryptography to secure itself in a similar way to how HTTPS and certificates secure the HTTP protocol.

It means that at each stage you can trust if a child's zone has the trust of a parent's zone and you can verify that the data contained within that zone hasn't been compromised.

Now another really critical part of DNSsec to understand before we touch upon why it's needed is the fact that it's additive.

It adds to DNS it doesn't replace it.

It's more accurate to think of any queries that you perform as either using DNS on its own or DNS plus DNSsec.

Conceptually imagine DNS at the bottom with DNSsec led on top.

Now in a situation where no DNS exploits have taken place it means that the results will be largely the same between DNS and DNSsec.

Let's say that we have two devices the one on the left is DNS only and the one on the right is DNSsec capable.

When querying the same DNS name servers the DNS only device will only receive DNS results.

It won't be exposed to DNSsec functionality and this is critical to understand because this is how DNSsec achieves backward compatibility.

A DNSsec capable device though this can do more than just normal DNS so it still makes queries and gets back DNS results but it also gets back DNSsec results and it can use these DNSsec results to validate the DNS results.

Assuming no bad actors have corrupted DNS in any way then this will go unnoticed since the results are the same but consider a scenario where we have changed DNS data in some way.

So again we have two devices the device on the left is DNSsec capable and the one on the right is standard DNS.

So the DNS only device performs a query and it thinks it's getting back the genuine website result only it isn't.

In an exploited environment it will be unaware that the result it gets the queries are bad and so the website it browses through might not actually be the one that it expects.

With DNSsec the initial query will occur in the same way and even though it's corrupt and will look valid what follows is that DNSsec will verify the result and because public private key cryptography is used together with a chain of trust architecture DNSsec will be able to identify that records have changed or they come from an origin which isn't the one that we're querying.

Now it's important to understand that DNSsec doesn't correct anything it only allows you to validate if something is genuinely from a certain source or not and if it's been altered or not it doesn't show you what the result should be but in most cases it's enough to know the integrity of something is valid or in doubt.

Now it might help you to understand one of the common risks posed by normal DNS if we step through it visually so let's do that.

Consider this architecture we have Bob on the left who's about to perform a query for the IP address of Categorum.io using this resolver server but we also have a bad actor Evil Bob at the bottom and in advance to perform this exploit he performs a query for Categorum.io this begins the process of walking the tree but while that's happening during this process where the resolver is walking the tree to get the true result Evil Bob responds with a fake response it pretends to be the real server so even while the real process is continuing Evil Bob enters false information into the resolver server in the middle and this result is now cash and the cash has been poisoned with bad data this means that when Bob queries for some Categorum.io records he's going to get the poisoned result this result is going to be returned to Bob and the effect of this is that Bob is going to be directed at something that he thinks is Categorum.io but isn't now this is just one way that DNS can be disrupted it's over simplified and there are some protections against it which illustrates how DNS isn't secure it was built during a time period where the internet was viewed as largely friendly rather than the adversarial zone which he now is from a security perspective now this point I just want to switch across to my command prompt and show you how a normal DNS query differs from a DNS sec query so let's go ahead and do that okay so we've moved across to my terminal and I'm just going to go ahead and use the dig utility which is a DNS utility and I'm going to perform a normal DNS query so this is the command dig its face www.icam.org and if I run at this query this is the result that we receive it's this answer section which I want to focus on now just to reiterate this is the query that I performed for this DNS name so in the answer section we have a result for www.icam.org and the result is that it's a C name and the C name points at another DNS record in this particular case www.vip.icam.org so directly below we can see this DNS record so www.vip.icam.org this time it's an A record an A records point at IP version 4 addresses and this is the IP version 4 address which corresponds to this DNS name and this DNS name maps back to our original query now because this is normal DNS we have no method of validating the integrity of this data you can see here that I'm querying this DNS server so 8.8.8.8 and this is not a DNS server that's affiliated with the ICANN organization so this result is not authoritative it's possible that this data is not valid either by accident or because it's been deliberately manipulated now DNSSEC helps us to fix this risk and let me show you how I'm going to start by clearing the screen and then I'm going to run the same query but using DNSSEC and I can do that using this command adding this additional option on the end of plus DNSSEC when I run this I receive both DNS and DNSSEC results so www.icam.org is a C name and it points at this DNS record slightly below it we can see the record that it points at this is an A record and just as with the previous query results it points at this IP version 4 address now what you'll notice is for each of these normal DNS results we also have this RRSEC and this is a DNSSEC resource type this is basically a digital signature of the record that it corresponds to and I'll show you in the next video how this digital signature can be used to validate the normal DNS data that's stored within this zone so we can query for normal DNS results and then validate the integrity of those results using DNSSEC now in this part of this lesson I just want to demonstrate exactly how a DNSSEC query result differs from a normal DNS result in the next video I'm going to expand on this and set you through exactly how these signatures work within a DNS zone at this point let's move back to the visual okay so what I just demonstrated is a way to avoid this kind of attack because even if a cache was poisoned a DNSSEC capable resolver would be able to identify the poison data and that alone is a huge improvement over standard DNS so at this point I hope you have a good idea of some of the ways which DNSSEC improves normal DNS that's it Identify the poisoned data and that alone is a huge improvement over-sanded at DNS.

So at this point I hope you have a good idea of some of the ways which DNSSEC improves normal DNS.

That's it for this video and the next one we're going to explore exactly how DNSSEC works in detail.

Now it's a lot to get through so I wanted to make sure that each different area of DNSSEC functionality has its own dedicated video.

At this point though, thanks for watching, go ahead and complete this video and when you're ready I look forward to you joining me in the next.

Welcome back, and in this video, which is part two of my DNS mini-series, I want to cover some of the reasons why DNS is structured in the way that it is.

Why do we need lots of DNS servers?

Why isn't one enough?

Additionally, at the end, I'm going to introduce some key DNS terms, and then introduce the hierarchical structure of DNS.

So let's jump in and get started.

Now, there are a few main problems with just having one DNS server, or even a small number of servers.

It's important that you understand these reasons, because it will help you understand why DNS is architected in the way that it is.

First, there's the obvious risk problem.

A small group of bad actors could attack the DNS infrastructure, and without much effort, prevented servicing genuine requests.

And that's something that we have to avoid for critical systems on the internet.

Also, we have a scaling problem.

Almost everyone who uses the internet globally uses DNS.

This represents a massive and growing load on the system.

A single server or a small group of servers can only get so big.

If every access is being made against one or a small group of servers, no matter what information is being requested, the system cannot scale.

Now, additionally, DNS is a huge database.

Current recent estimates predict that there are around 341 million domains, such as Netflix.com, Apple.com, and Twitter.com.

And each of those domains might have many records, tens, hundreds, thousands, or even more records in each of those domains.

And so this represents a huge data volume problem.

We can start with the amount of data, but then also need to take into consideration updates to that data, as well as consistency issues.

And all of that data is accessed by anyone using the internet on a constant basis.

Now, we can address the risk problem by creating more servers, each of them storing an exact copy of the same DNS data.

The more servers we have, the more load can be tolerated against those servers, and the less risk of attackers managing to take down the entire platform.

But this method doesn't really do anything about the scaling problem.

If every user of DNS communicates with any of the servers at random, it means that every server needs to hold the complete DNS dataset.

And this is a huge amount of data.

It's a huge global scale monolith, and this is something that we need to avoid.

Ideally, we also need to have the ability to delegate control over certain parts of the DNS dataset to other organizations so that they can manage it.

So UK domains, for example, should be managed by a UK entity.

US domains should be managed by somebody in the United States.gov, by the US government.au, by an organization in Australia, and so on.

And for this, we need a hierarchical structure.

I'm going to be talking about this fairly soon, but for now, I need to introduce some DNS terms.

Now, I need you not to switch off at this point in the video.

I'm going to be using some bullet points.

I hate bullet points, but these are worth it, so please stay with me.

The first term that I want to introduce is a DNS zone.

Think of this like a database.

So we have Netflix.com, and that's the zone.

Inside that zone are DNS records, for example, www.netflix.com, as well as many others.

Now, that zone is stored on a disk somewhere, and that's called a zone file.

Conceptually, this is a file containing a zone.

So there's a Netflix.com zone file somewhere on the internet containing the Netflix.com zone and all of its records.

We also have DNS nameservers, known as NS for short, and these are DNS servers which host one or more zones, and it does so by storing one or more zone files.

It's the nameserver or nameservers of the Netflix.com zone, which can answer queries that you have about the IP address of www.netflix.com.

Next, we have the term authoritative, and this just means that for a given domain, this is the real or genuine, or to put another way, the boss for this particular domain.

So there are one or more nameservers which can give authoritative answers for www.netflix.com.

These can be trusted.

They're the single source of truth for this particular zone.

And we also have the opposite of this, which is non-authoritative or cached, and this is where a DNS server might cache a zone or records to speed things up.

Your local router or internet provider might, for instance, be able to provide a non-authoritative answer for learn.control.io or youtube.com, because you've visited those sites before.

But only my nameservers can give an authoritative answer for learn.control.io.

Now that you know those terms, I want to introduce the architecture of DNS, so it's hierarchical structure.

And don't worry, this is high level only.

I'll be covering the detail of how DNS works and how it's used in a follow-up video.

So at this point, you know why a single DNS server is bad.

You also know why having many DNS servers is bad if they just saw the same monolithic set of data, and you understand a few important DNS terms.

What I'm going to step through now is a hierarchical design, which is the way that DNS works.

Using this architecture, you can split up the data which DNS stores and delegate the management of certain pieces to certain organizations.

Splitting the data makes it easier to manage and also splits the load.

If you're doing a query on netflix.com, you generally won't have to touch the infrastructure at the twitter.com.

Now DNS starts with the DNS root, and this is a zone like any other part of DNS, and this zone is hosted on DNS name servers also just like any other part of DNS.

So the DNS root zone runs on the DNS root servers.

The only special element of the root zone is that it's the point that every DNS client knows about and trusts.

It's where queries start at the root of DNS.

Now there are 13 root server IP addresses which host the root zone.

These IP addresses are distributed geographically, and the hardware is managed by independent organizations.

The internet corporation for assigned names and numbers, or ICANN, operates one of the 13 IP addresses which host the root zone.

And others include NASA, the University of Maryland, and Verisign.

So to be clear, these organizations manage the hardware for the 13 DNS root server IP addresses.

In reality, each of these 13 IP addresses represents many different servers using anycast IP addresses.

But from DNS' perspective, there are 13 root server IP addresses.

Now the root zone, remember, this is just a database.

This is managed by the internet assigned numbers authority known as IANA.

So they're responsible for the contents of the root zone.

So management of the root zone and management of the root servers which host the root zone is different.

Now the root zone doesn't store that much data.

What it does store is critical to how DNS functions, but there isn't that much data.

The root zone contains high level information on the top level domains or TLDs of DNS.

Now there are two types of TLD, generic TLDs such as .com and country codes specific ones such as .uk and .au.

IANA delegates the management of these TLDs to other organizations known as registries.

Now the job of the root zone really is just a point at these TLD registries.

So IANA delegates management at the .com TLD to Verisign, meaning Verisign is the .com registry.

And so in the root zone, there's an entry for .com pointing at the name servers which belong to Verisign.

Now .com is just one TLD.

There are other entries in the root zone for other TLDs and other TLDs could include .io, .uk and .au and many more.

Because the root zone points at these TLD zones, they're known as authoritative the source of truth for those TLDs.

This process where the root zone points at the name servers hosting the TLD zones, it establishes a chain of trust within DNS.

So to summarize, the root zone is pointing at the name servers hosting the TLD zones run by the registries which are the organizations who manage these TLDs.

So Verisign will operate some name servers hosting the .com TLD zone and the root zone will have records for the .com TLD which point at these .com name servers.

The .com zone which is just another DNS zone also contains some data, specifically high level data about domains which are within that .com TLD.

For example, the .com TLD zone contains some records with Twitter.com and Netflix.com, so records for domains which exist inside the .com zone.

The TLD only contains this high level information on domains within it, for example, Netflix.com.

It doesn't contain detailed records within these domains, for example, www.netflix.com, all the TLD contains is information on the domain itself.

Specifically, with this example, a set of records for Netflix.com which point at the name servers which host the Netflix.com zone.

Now it will also contain records for Twitter.com which point at the name servers which host the zone for Twitter.com as well as records for every other domain within the .com TLD.

Now these name servers, because they're pointed at from the layer above, they're authoritative for the domains, the zones that they host.

So the name servers for Netflix.com are authoritative for Netflix.com because the Netflix.com entry in the .com TLD points at these name servers.

Now these name servers host the zone for a given domain, for example, Netflix.com.

This means the servers host the zone file which stores the data for that zone.

At this level, the zone contains records within Netflix.com, so www.netflix.com which points at a set of IP addresses.

And because the zone and zone files are on these name servers and because these name servers are authoritative for the domain, these zones and zone files are also authoritative.

Now don't worry about understanding this in detail.

In the next video, I'm going to be walking through how this works in practice.

For now, all I need you to understand is that each layer of DNS from the root, the TLDs and the domain name servers they store.

[POP] Or only a small part of the DNS database.

The root zone knows which name servers the .com zone is on, the .com zone knows which name servers Netflix.com zone is on, and the Netflix.com zone contains records for the Netflix.com domain and can answer queries.

So this is the hierarchical architecture of DNS.

And in the next video, in this video series, I'm going to be stepping you through the flow of how DNS works and discussing the architecture at a more technical level.

But at this point, that's everything I'll be covering in this video.

So go ahead and complete the video.

And when you're ready, you can join me in the next video of this series.

Well, welcome to the first video in this series where I want to help you understand DNS.

DNS is one of the core services on the internet, and it doesn't work, applications and other services will fail.

Now in this video series I'll be covering what DNS does, why it's structured the way that it is, how DNS works to get us answers to queries, and I'll finish up by covering some of its key limitations.

Now with that being said, let's jump in and get started.

Now before I cover how DNS works and why it works in the specific way that it does, I want you to be 100% sure of what functionality DNS provides.

Now when you access any website, you type the name into your browser, for example www.netflix.com.

Now you might imagine that the name is used to connect to the Netflix.com servers and stream your movie or TV show, but that's not actually how it or any internet app generally works.

Simply put, humans like names because they're easy to remember, but networks or servers not so much.

To communicate with Netflix, your computer and any networking in between needs the IP addresses of the Netflix servers.

DNS actually does many different things, but at its core it's like a big contact database.

In this context, it links names to IP addresses, so using DNS when accessing Netflix, we would ask DNS for the IP address of Netflix.

It would return the answer and then our device would use that IP address to connect over the internet to the Netflix servers.

So conceptually, the main piece of functionality which DNS provides is that it's a huge database which converts DNS names from Netflix.com into IP addresses.

Now so far, I hope that this makes sense.

At this point, it sounds just like a database and nothing complex, and you might be asking yourself why not just have one DNS server globally or a small collection of servers?

Now we're going to review that in the net.

Next video.

For this video, I just wanted to set the scene and make sure you understand exactly what functionality DNS provides.

DNS is critical.

Many large-scale failures on the internet are caused by either failed DNS or badly implemented DNS infrastructure.

If you want to be effective at designing or implementing cloud solutions or network-based applications, you have to understand DNS.

So if you are interested in knowing more, then go ahead and move to the next video, where I'll cover why DNS is structured in the way that it is.

Welcome back and in this lesson I want to talk about digital signing or digital signatures.

This is a process which you need to be familiar with to understand many other areas of IT such as DNFSEP or SSL certificates.

If you haven't watched my video on hashing it's linked in the description and you should pause this video and go and do that now because it's required knowledge for this video.

At this point though let's jump in and get started straight away.

Now before I cover how digital signatures work in detail I want to give you a quick refresher on public key cryptography.

With public key cryptography you generate two keys as part of a pair.

We have on the left the private key and this part is secret and should only ever be known by the owner.

Then we have the public key and this is not secret.

In fact anyone can know it or possess it.

It should be public and ideally as widely distributed as possible.

Now these keys are related, they're generated as part of one operation.

It means that you can take the public key which remember anyone has or anyone can get and use it to encrypt data which can then only be decrypted by the matching private key.

The data can't even be decrypted using the same public key which encrypted it so it allows you to securely send data to the owner of the private part of the public key.

What this architecture also allows is that you can take something and sign it using the private key and think of this just like encryption but where anybody who receives the data and has the public key can view the data and verify that the private key part was used to sign it and this provides away the evidence that you are in control of the private key used to sign some data since the private key is secret and since only you should have it.

This provides a way to establish a form of digital identity or digital validation.

This signing architecture is important because it forms the foundation of many other IT processes.

The process of adding digital signatures to data adds two main benefits when used in conjunction with hashing.

It verifies the integrity of the data that the data you have is what somebody produced and it verifies the authenticity so the data that you have is from a specific person.

So integrity is what and authenticity is who.

Together it means that you can download some data from Bob and verify that Bob produced the data and that it hasn't been modified.

Nobody can falsify data as being from Bob and nobody can alter the data without you being able to tell.

Now a key part of this process is that it's led on top of normal usage so if somebody doesn't care about integrity or authenticity they can access data as normal without any of these checks and to enable that the first step is to take a hash of the data that you're going to be validating.

The original data remains unchanged in plain text or whatever its original format is.

If you don't care about the integrity and authenticity then you can use the application or consume the data without worrying about any of this process.

Now this means that anybody having both the data and the hash knows that the data is the original data that Bob produced assuming that they trust the hash to be genuine and that's what digital signatures enable.

Next Bob signs the hash using his private key and this authenticates the hash to Bob.

Bob's public key can be distributed widely onto many locations that Bob controls and using this public key you can access the signed hash.

Because of this you know it came from Bob's private key i.e.

Bob and so the hash is now authenticated as being from Bob.

Nobody can falsify a fake hash because only one person Bob has Bob's private key.

So now we know the hash is from Bob we can verify that because we have Bob's public key.

We know the hash can't be changed because only Bob has Bob's private key and only the private key can sign anything and appear to be from that private key.

We now have this authenticated hash and we verified the integrity because of the private-public-key relationship.

We also have the original document and we know that it's authentic because if it was changed then the hash wouldn't match anymore.

So if we trust Bob's public key then we know that anything being signed by his private key is authentic because we know that Bob and Bob only have this private key then we trust the entity i.e.

Bob and because Bob can now digitally sign a hash we know that any data that we receive from the Bob is both authentic i.e.

Bob has authored this data and it's not been changed during transit or download.

So we have this chain of trust and this chain of trust using public key cryptographic signing and hashing forms a foundation and many things within id which you take for granted.

Okay so now that you understand the basic building blocks of this process let's step through this visually.

So step one is Bob and his private key.

Bob is the only person to have his private key but he's uploaded his public key to his website, his twitter account, various public key retro services and he includes a link to his public key in all of the emails that he sends.

Now if you look at all of these copies of his public key if all of them match then you can assume that they're valid and from Bob.

To exploit this you have to take over all of these services at the same time and change all mentions of his public key so the wider the distribution of the public key the easier it is to spot if any of them have been modified.

So let's say that Bob creates a contract to send it to a business partner and he wants others to be able to verify firstly that he sent it and that he wasn't altered in transit.

So the next step is that he puts the document through a hash function which results in a hash.

The hash as you've learned is unique to this document.

Know all the document can have this hash and any changes to this document also change this hash and you can't derive the document from the hash.

What we can't prove yet is that Bob created this hash or that the hash hasn't changed as part of faking the document.

So to fix this Bob uses his private key and he signs the hash and this creates a signature and he bundles this signature with the original document which creates a signed document.

So this signed document is the original document so the data plus a copy of the signed hash of that document and that data is going to be hosted somewhere so let's say that Bob uploads it to the internet or he emails the document to somebody or stores it from some cloud storage.

Now next one of Bob's business partners downloads the contracts of the signed data and the first thing is to get Bob's original hash and so to do that we take the signature and Bob's public key and that gives us back Bob's hash.

So now we know that this hash is signed by Bob and we know that Bob created this hash and this means that we know what Bob thought the hash of the document was.

We know the original state of the document when Bob generated the hash.

So we take the document and we hash it with the same hash function as Bob used.

So now we have our hash of the document and we have Bob's original hash verified to be from Bob.

Now these two hashes match you know that the document that you have is the document that Bob generated.

It hasn't been altered and you know that it originated from Bob because his hash was signed using his private key to generate the signature which is part of this document which is digitally signed and this is how hashing together with public key cryptography specifically signing can be used to verify authenticity and integrity.

Now Bob could have taken this a step further and encrypted all of this with a public key at the intended recipient and ensured that all of this process could happen in a fully encrypted way but encryption and signing are two slightly different things which are both enabled by the same public key cryptography architecture.

Now this point has everything I wanted to do.

I just wanted to give you a really high-level overview of how digital signatures can be used to verify both the integrity and the authenticity of data.

Now you're going to be using this knowledge as you learn about all that important IT processes so it's really important that you understand this end-to-end but at this point that is everything I wanted to cover in this video so go ahead and complete the video and when you're ready I look forward to you joining me in the next.

Welcome back and in this lesson I want to talk about hashing, what it is and why we need it.

Now we do have a lot to cover so let's jump in and get started straight away.

What's simply hashing is a process where an algorithm is used to turn this or any other piece of data into this, a fixed length representation of that data.

Hashing is at the core of many critical services that you use today, such as passwords, digital signatures, SSL certificates, data searches, and even some antivirus or anti-malware solutions rely on hashes so they can store definitions of malicious files without having to store every single huge file.

We even have some forms of digital money such as Bitcoin which use hashing.

Now to understand why hashing is so important we need to step through how it works, what benefits it provides and some of the terminology used.

So let's go ahead and do that.

Hashing starts with a hash function and think of this as a piece of code and algorithm.

Now there are many different ones which are used or have been used over the years.

An example include MD5 and SHA-2 256.

The core principle of hashing is that you take some large variable sized data and you put that data through a hashing function and receive a fixed size hash of that data.

So whether the data is a text file, an image or a huge database file, the hash you receive will be tiny and fixed length based on the hashing function type.

Now also critically if you take some other data and again put it through the same hashing function you will get a different hash, a unique hash value.

Even if the data differs by only one byte, one character or one pixel it will result in a different hash.

The aim with hash functions is that any change no matter how minor will result in a different hash value.

Another critical part of hashing is that while getting a hash from some data is trivial what you cannot do is take a hash value and get the original data back.

There's no way to do this.

Let's say that you had a hash of an image.

Well you couldn't take the hash and derive the image.

You could if you had infinite processing power and infinite time brute force every image in existence to try and link back to the hash.

But this would require hashing every single image until you found the correct one.

You should view it as impossible with any modern hashing algorithm to derive the original data from a hash.

Without some vulnerability in the hashing function or without infinite time and processing power the hashing is one way only.

Data through to hash.

Now lastly another fundamental part of hashing is that given the same data you're always going to get the same hash value if you use the same hashing function.

So for one piece of data you get one hash value, for a different piece of data you get a different hash value and hashing is one way only.

And you should never get the same hash value for different data.

There are more on this in a second.

Let's look at an example of where hashing can be used which should be pretty familiar.

Imagine you're using an online service and they have a single server.

And when you create an account on that service you create a username and password and both of those are stored on that server.

When you log in you send your username and password to the server and an automated process checks the password that you send with the one that's stored and if they match you can log in.

Now even if you encrypt your password in transit and even if the password is encrypted when stored on the server your password still exists on that server.

And it means if the server is ever exploited then a bad actor will have access but worst to your password and at best an encrypted version of your password.

Assuming a full data dump or a long term exploit is pretty trivial to get the plain text version of your password in this way.

If you use that password on all the services those services are also at risk.

But what about if we use hashes?

Well with this architecture instead of sending a password to the server when signing up or signing in we send a hash of the password.

This means the server instead of having our actual password it only stores the hash of our password.

All the server needs to do is check that the hash that you send matches the one in its database and it can confirm the password was entered correctly on the local client.

Because given the same data in this case password and the same hashing algorithm you'll end up with the same hash value.

So by comparing the hash value it stores to the hash value that you deliver by the hash of your password it can check you're entering the correct password without ever storing a copy of your password.

Now in this example I've used the MD5 hashing algorithm but as you're going to say in a second this isn't super secure anymore so you'd likely need to use another hashing algorithm.

I'm going to use MD5 as an example to demonstrate a weakness of some algorithms.

So to stress you wouldn't generally use MD5 for production password systems you'd use something a lot more secure.

Now if this server was ever exploited when using password hashes this would be much safer because the hashes are one way you can't derive the password from a hash.

Nothing stops the attacker though from getting over and over again trying every possible word and phrase combination with the hashing algorithm used until it gets us right.

And then it has confirmed the password that you used and it can try and exploit all the services which you also make use of.

And this is why it's really important to use a modern secure hashing algorithm.

Now there are two things with hashing which are really bad.

One is if we were ever able to take a hash and derive the original data but as I mentioned earlier that's basically impossible to have a critical vulnerability in how hashing works.

Another major problem would be a collision.

An example of a collision is that if we take this image of a plane and if we hash this image and then if we take another image say this image of a shipwreck and we also hash this image we should have two different hash values.

If we do awesome.

If not if the hash of A on the left equals the hash of B on the right then bad things happen because we can no longer trust the hash function i.e. the hash algorithm.

And this is one of the reasons that MD5 has hashing algorithm is less trusted because collisions can happen.

We can actually show how they can be created, how data can be manipulated to cause collisions.

Now I've attached some links to this video with the research project showing how we can create those collisions.

But as a quick example I want to switch to my terminal and demonstrate how this works with these two images.

So in this folder on my local machine I have two images plane.jpg and ship.jpg and those represent the two images that you've just seen on your screen moments ago.

I'm going to go ahead and generate a hash value of one of these files and I'm going to use the MD5 hashing algorithm.

So I'm going to put MD5 space and then plane.jpg.

Go ahead and focus on the hashing value that's been generated so this hashing value in theory should uniquely represent the image plane.jpg.

So plane.jpg should always generate this hash value and if I repeat this command I get the same hash value.

But what should happen is if I generate a hash of another piece of data I should get a different hash value.

So I'm going to run MD5 again this time on ship.jpg.

Watch what happens in this case it's the same hash value and this is an example of a collision where two different pieces of data generate the same hash value.

And this is a weakness of MD5.

We can create this collision.

We can adjust the data in order to generate this collision and this is a bad thing.

This shouldn't happen.

Now I'm not to follow the same process but using a more secure hashing algorithm.

So this time I'm going to use the SHA-2 256 algorithm on the same file plane.jpg.

Now watch what happens now.

The hash value is longer because this is a different hashing algorithm but we confirm that this hash value is for plane.jpg.

Now I'm going to run the same hashing algorithm on ship.jpg.

This time note how it's a different hash value.

This is a much more secure hashing algorithm.

SHA-2-256 is much better at protecting against these collisions.

It's a more modern and more well trusted hashing algorithm.

Now just like any other form of security such as encryption it's important that you always use the latest and most secure protocols and architectures.

So if you're using hashing in production you should generally use something like SHA-2-256 because if you want to guarantee that one-to-one link between a piece of data and a hash so that any other piece of data generates a different hash you need to make sure you're using a well respected hashing algorithm such as SHA-2-256.

Now the likelihood of this happening in normal usage is nearly possible because these two images have actually been artificially adjusted to cause this collision but it does represent theoretical vulnerability in the MD5 hashing algorithm.

I've included links attached to this video which detail the search project and some examples of how you can implement this as a personal project if you want.

But at this point I'm going to go ahead and return to the remainder of this video.

Now just to summarise with hashing you take some data plus a hashing function and you generate a hash value.

Now you can't get the original data from a hash it is a one-way process and the same data should always generate the same hash.

Different data should always generate a different hash.

Now just demonstrated how you can artificially cause a collision using older hashing algorithms but in the real world even older algorithms should generate a different hash for different data and any modern hashing algorithm is protected against even this artificial process.

Now hashing can be used to verify downloaded data.

If you're making some data available to download you can have the download in one location and the hash of that download stored on a different system.

It means that you can download the data you can hash it and generate your own hash value.

If the hash value is matched then the downloaded data hasn't been altered.

If they differ then it means that what you have is not the same data as what was made available by the original author.

And this is a process that's very often used to verify sensitive applications.

So if you're downloading any form of application which stores sensitive data or operates on sensitive networks then you'll generally find that a hash will be made available by the author of that application and it can be used to verify that that download has not been adjusted.

Now in these type of security sensitive situations or if you're a security professional you also need to be sure that the hash itself hasn't been altered.

It's also whether the hash itself was generated by the person who claims to have generated that hash.

So if I make some software available to you and you download it you need to first check that the download hasn't been altered by hashing it yourself and comparing your hash to the hash that I publish.

But you also need to be sure that it was me publishing that hash and that the hash that you download hasn't been altered in some way.

And a way that this can be done is using digital signing or digital signatures and this is something that I cover in another video.

But at this point that's everything I wanted to cover in this video so go ahead and complete the video and be ready.

I'll look forward to you joining me in the next.

Now another process which uses asymmetric keys is signing.

Let's review an example.

The robot general wants to respond to the battle plan.

So let's say that the robot general has received the battle plans from the cap ruler, and he wants to confirm A that he's received them, and also that he agrees to them.

Remember, the battle plans require both sides to operate as one, and so the cap ruler needs to know that the robot general has received the plans, and that he agrees with them.

The general might want to respond with a simple OK message.

So we'll send the message saying OK to the cap ruler.

The issue is that anyone can encrypt messages to another party using asymmetric encryption.

Your eye could get hold of the cap ruler's public key and encrypt a message saying OK, and send it to the cap ruler.

And the cap ruler wouldn't necessarily be aware whether that was from the robot general or not.

Just because the cap ruler gets a message from what appears to be the robot general saying OK, it doesn't mean that it's actually from the robot general.

It could be from a human pretending to be the robot general.

Encryption does not prove identity.

But what we can use is a process called signing.

With signing, the robot general could write this OK message, and then he could take that message, and using his private key, he can sign that message.

Then that message can be sent across to the cap ruler.

And when the cap ruler receives that message, he can use the robot general's public key to prove whether that message was signed using the robot general's private key.

So this is the inverse.

On the previous example, I demonstrated how you can use the public key to encrypt data that can only be decrypted with the private key.

In this example, we can take the robot general's private key and sign a document.

And then the public key of the robot general can verify that that document was signed using its matching private key.

At no point is the private key revealed.

It's just because of this relationship between the public and private key.

The public key can verify whether its corresponding private key was used to sign something.

And so signing is generally used to verify identity.

As long as we can be sure that the public key belongs to the robot general, and generally this is done by the robot general uploading his public key to his Twitter account, or putting it on his cloud storage or his website.

As long as that verification exists, we can get the public key and verify that a document has indeed been signed by his private key because of that relationship between these two keys.

So key signing is generally used for ID verification and certain logon systems.

There's one more thing I want to talk about before I finish up this lesson.

And that's steganography.

Sometimes encryption isn't enough.

The problem with encryption is that if you use it, it's obvious that you've used it.

If you encrypt a file and deliver it to me, there isn't really any scope for denying that you have encrypted some form of data.

The government, who control the men and women with guns, can often insist that you decrypt the data.

And if you don't, well, they have plenty of sticks so they can put you in jail.

You can refuse to decrypt the data, but the deniability isn't there.

If you encrypt data, somebody will know that you've encrypted data.

Now, steganography is a process which addresses this.

If you've ever used invisible ink, the kind which only shows under a certain type of light or when heated, that's a physical form of steganography.

It's a method of hiding something in something else.

With steganography, the cap ruler could generate some ciphertext and hide it in a puppy image.

The image could be delivered to the robot general who knows to expect the image with some data inside and then extract the data.

To anyone else, it would just look like a puppy image.

And everybody knows there's no way that the cap ruler would send the robot channel a puppy image to this plausible deniability.

The effect of steganography might be a slightly larger file, but it would look almost identical.

Effective steganography algorithms make it almost impossible to find the hidden data unless you know a certain key, a number, or a pattern.

Steganography is just another layer of protection.

The way it works at a foundational level is pretty simple.

Let's say that we wanted to hide a simple message, just high.

Well, the decimal values for h and i are 8 and 9.

So we might take the puppy image and pick two random pixels and change the color by 8 and 9 respectively.

The steganography algorithm would take the original picture, selects the required number of pixels, adjust those pixels by a certain range of values.

And what it would generate as an output would be an almost identical puppy image.

But hidden there would be slight changes.

If you don't believe me, let's blow this up a little bit because this is an actual simple example of steganography.

If you look really closely at where those arrows are pointing, the color is slightly different than the background.

The first pixel has been adjusted by eight values and the second has been adjusted by nine values.

And so if you knew the location of these two pixels, you could take the second image and extract the text.

Now, this is a super simple example.

A real algorithm would be much more complex.

But this is at base level how the process works.

It allows you to embed data in another piece of data.

To be really secure, the cap roller would encrypt some data using the robot general's public key, take that cypher text, use steganography to embed it in an image that wouldn't be tied back to the cap roller, send this image to the robot general, and then the robot general could also use steganography to extract the piece of cypher text and decrypt it using his private key.

And the same process could be followed in reverse to signal an OK.

But the robot general, in addition to encrypting that OK, would also sign it so the cap roller would know that it came from the robot general.

With that being said, go ahead, complete this video.

When you're ready, you can join me the next.

I agree with this statement. I am not sure if I would consider myself a polyglot yet, but someday, I would love to be one. Being bilingual has given me so many opportunities and experiences that make me want to learn more languages. Knowing that theirs more experiences for me in life by learning more languages and, thus, culture excites me a lot and passionates me. So, even if it's just a word or two, I always want to learn more because I love languages and language learning.

Welcome to this video where I want to talk at a high level about hardware security modules known as HSMs.

Now these are a really important type of device to understand both in general and especially if you currently work in or want to work in the security space because so many other things rely on them to function.

Now let's jump in and step through why we need them, what they are and how they work.

Now let's start by looking at the world without hardware security modules and we're going to do that with the example of a virtualized environment and so we have a VM host and this could be VMware, Zen or something within a cloud environment.

It doesn't matter for this example.

This means that we have some physical CPUs, memory as well as mechanical or solid state storage.

The reason on this is our hypervisor, a pair of operating systems and to keep this simple, a pair of applications.

Now, only to imagine that these applications, the operating systems and the hypervisor are all using encryption in some way.

That might be encryption at rest or in transit, it might be public key infrastructure or it might be simple SSL or TLS.

Whatever the requirement, it means you're going to have keys stored in many places, keys inside the applications, controlled by the operating system or held by the hypervisor.

And all this means that keys will be handled by the CPU, held in memory and stored on storage.

And over time, if you care about disaster recovery, you're going to have keys stored on various backups, some of which might go offsite for storage.

Using encryption means using keys and these keys will be stored or held in various places.

You might think this is controllable, but over time they will leave your premises and because of this, your directs control, meaning it becomes easier for these to fall into the wrong hands and become exploited.

Now, that's where HSMs add value.

So let's take a look.

Now, this is a similar architecture, the same hypervisor, the same set of operating systems and applications and the same backup infrastructure.

Only now, we've chosen to utilize a HSM.

A HSM or hardware security module, as the name suggests, is a separate device or cluster of devices.

It's isolated from your main infrastructure and it's inside this device that your keys are stored.

They never leave this device.

They're managed by this device, often generated and deleted by this device.

Anytime you want to perform cryptographic operations, you send those to the HSM together with the data.

The HSM performs cryptographic operations and sends the result back.

It means you keep the same application and virtualization architecture, but instead of having to generate, to manage, store and secure keys and risk those leaking, with HSM, the keys are securely held on device.

So that's HSMs at the high level.

Let's finish by exploring the architecture in more detail.

So now we have our HSM in the middle.

Think of this as 100% separated from the rest of our infrastructure, accessible only in a highly defined way.

Keys are created on the HSM, stored on the HSM, operations happen on the HSM and keys generally never leave the device.

By utilizing HSMs, you create a secure island within your infrastructure, where all cryptographic operations are controlled from.

The authentication takes place inside the device.

This means you have an isolated security blast radius.

Even if your corporate identity store is exploited, the identities used within the HSM are internally defined, and so can withstand this type of exploit.

HSMs are tamper-proof and they're hardened against physical and logical attacks.

The device is used secure on-glaves internally, which makes it almost impossible to gain access to the internal key material through direct physical means.

Many smartphones today come with a similar cut-down version of this.

It stores your biometric information to keep it isolated from any badly behaving software on your smartphone.

Access to cryptographic operations within the HSM is tightly controlled.

You need access, but assuming you have those access permissions, this access is still via industry-standard APIs, such as PKCS11, JCE and CryptoNG.

Nothing is directly accessible.

It's all controlled via APIs.

Now, there's even role separation for admins, so you can define people who can admin the HSM for things like software updates, key generation and other admin tasks, but those people might not be able to perform cryptographic operations.

Many HSMs are audited to some very stringent standards, such as those required for US government use, and it's this auditability, this access control, which makes them such a powerful type of device.

Examples of the types of situations where you might use a HSM, and this is a very small subset there are many others, but you might use it to off-load processing for SSL or TLS onto the HSM.

So if you have a fleet of web servers, you might have the HSM device perform heavy lifting on your behalf instead of the web servers.

HSMs often handle this in hardware using acceleration, so you gain the benefits of secure key management and the performance boost via off-loading.

You might also use HSMs for signing certificates for a private PKI infrastructure that you have within your business.

This just provides a way that you can securely manage the key material used to sign your certificates.

Now, I wanted to keep this video brief and just provide a very high-level introduction, because I'm going to be making many more videos in this series.

In order for those to make sense, you need to understand why HSMs are needed, what they do, and how they work at a high level.

And so that's what I covered in this video.

Now, I hope you've enjoyed it, but that's everything for now.

So go ahead and complete the video, and when you're ready, I look forward to you joining me in the next.

Welcome to this lesson where I want to provide a quick foundation into encryption.

Now I want to keep foundation lessons as short as possible so let's jump in and get started.

Before we get started though, I just want to cover the topics that we're going to go through in this lesson.

I'll be starting by talking about the different approaches to encryption, so encryption at rest and encryption in transit.

I'll follow up by talking about the different concepts, so the different components and how those fit together.

I'll cover symmetric encryption, asymmetric encryption, including the differences between those two, and I'll finish up the lesson talking about signing and then steganography.

Now I'll get started by talking about the different approaches to encryption, so we'll do that first.

There are two main approaches to encryption that you will see used within AWS and the real world in general.

Each of these is aimed at solving a very different problem.

First we've got encryption at rest and second encryption in transit.

Encryption at rest is designed to protect against physical theft and physical tampering, and a common example of this is a user with an encrypted laptop.

So Nat's using her laptop as she would with any other device, but her laptop is busy encrypting or scrambling any data that it writes to the internal storage, and then decrypting that data when it reads it from the same storage into memory.

Now there's a special piece of data that's used to encrypt and decrypt that data, and it's only known to Nat.

Now the proper word for this is secret.

Now with laptop encryption, this is either the password for the user logging into the laptop, or a piece of data that's derived from that, but in other types of encryption, it's more complex than that.

What this means though, is that if Nat's laptop is stolen or tampered with, the data is encrypted at rest without the information required to decrypt it.

It's useless to an attacker.

If somebody steals a laptop without the passcode that Nat uses, all they have is a laptop with encrypted or scrambled data, which is useless.

Encryption at rest is also used fairly commonly within cloud environments.

Your data is stored on shared hardware, and it's done so in an encrypted form.

Even if somebody else could find and access the base storage device that you were using, they couldn't access your data.

Encryption at rest is generally used where only one party is in this case, Nat, and that party is the only person who knows the encryption and encryption team.

The other approach to encryption is known as encryption in transit, and this is aimed at protecting data while it's being transferred between two places.

So when Nat is using her encryption data, the data is encrypted before it exits Nat's laptop, and decrypted by the bank when it arrives, and the same process is followed in reverse.

So the bank encrypts any data that's destined for Nat's laptop, and Nat's laptop performs the decryption process.

What you're essentially doing with encryption in transit is to apply an encryption wrapper, a tunnel, around the raw data, and anyone looking from the outside would just see a stream of scrambled data.

Encryption in transit is generally used when multiple individuals or systems are involved.

So let's move on and talk about encryption concepts.

In this part of the lesson, I want to introduce some encryption terms.

Not all of these are immediately intuitive, and so if you haven't heard of these before, I want to confirm your understanding because I'll be using them throughout the course.

Now we'll start with plaintext, and this is a horrible term to use for this thing, because the name gives you the impression that it's text data, and it isn't always.

Plaintext is unencrypted data.

It can be text, but it doesn't have to be.

It can also be images or even applications.

Plaintext is data that you can load into an application and use, or you can load and immediately read that data.

The next term is an algorithm, and an algorithm is a piece of code, or more specifically a piece of maths which takes plaintext and an encryption key, which I'll talk about shortly, and it generates encrypted data.

Now common examples of algorithms are Blowfish, AES, RC4, DES, RC5, and RC6.

When an algorithm is being used, it needs the plaintext, and it needs a key.

And a key is the next term I want to talk about.

A key at its simplest is a password, but it can be much more complex.

When an algorithm takes plaintext and a key, the output that it generates is ciphertext.

Now just like plaintext, ciphertext isn't always text data.

Ciphertext is just encrypted data.

So the relationship between all these things is that encryption, it takes plaintext, it uses an algorithm and a key, and it uses those things to create ciphertext.

Decryption is just the reverse.

It takes ciphertext, it takes a key, and it generates plaintext.

Now this is not all that complex at a high level, but like most things in tech, there are some details which you need to understand.

First I want to focus on the encryption key for the next part of the lesson.

The type of key influences how encryption is used.

So let's look at the different types of keys and different types of encryption.

The first type of encryption key that I want to talk about is a symmetric key.

Symmetric keys are used as part of a symmetric encryption process.

Now it's far easier to show you an example visually rather than just explain it.

So here goes.

Now as everybody knows at this point I'm a fan of animals, specifically cats.

What you might not know is I'm also a fan of robots.

And everybody knows that cats want to achieve world domination, and robots are working towards the robot apocalypse.

In this example, they've allied.

They created a plan for world domination.

So on the left we've got the cat supreme ruler, and on the right we've got the robot general.

Both leaders want to exchange data, their battle plans, and they want to do that without humans being able to read them in a plaintext form.

They need to ensure that the battle plans are only ever exchanged using ciphertext, so the humans never see the plaintext battle plans.

So step one is they agree on an algorithm to use, in this case AES 256.

And they set to work preparing to send the plaintext battle plans.

Now the cat ruler, because he's the party sending the data, he needs to generate a symmetric encryption key, so he needs to create that and keep it safe.

A symmetric encryption algorithm is used, and this accepts the key and the plaintext battle plans.

And once it's accepted both of those, it performs encryption and it outputs ciphertext, the encrypted battle plans.

The encrypted battle plans are now secure, because they're ciphertext and nobody can decipher them without the key.

They can be sent over any transmission method, even an insecure way to the robot general.

The encryption removes the risk of transmitting this data in the open, so even if we handed the ciphertext over to an untrustable party and asked for him to deliver that to the robot general, that would still be safe because the ciphertext is un-desyferable without the key.

But this is where the problem starts for our rulers.

The robot general doesn't have the key which was used to encrypt the data.

With symmetric encryption, the same key is used for both the encryption and the encryption processors.

So we need to find a way to get the robot general a copy of the key that was used to encrypt the data.

So how do we do that?

Well, we could transfer it electronically, but that's a bad idea because if the humans get the key, it's all over.

They can also decrypt the data.

We could arrange an in-person meetup, but for anything which is really sensitive, this is less than ideal because the people meeting to exchange the key could be intercepted on their way.

We could encrypt the encryption key and then transfer that key.

Now, that would be safe because the encryption key would be protected, but we'd still need to find a safe way of transferring the key that was used to encrypt the encryption key, and that gets really complex really quickly.

This is why symmetric encryption is great for things like local file encryption or disk encryption or lac-box, but not so useful for situations where the data needs to be transferred between two remote parties, because arranging the transit of the key is the problem, and generally we need to do that in advance so there is no delay in decrypting the data.

If the data that we're transferring is time-sensitive, the transit of the encryption key needs to happen in advance, and that's the most complex part of this method of encryption.

Now, if we did have a way to transfer the key securely, then the same algorithm would decrypt the data using the key and the ciphertext, and then we'd return the original plaintext battle plans.

But there's another way of doing it, and that's to use asymmetric encryption, and this addresses some of the problems that our rulers are facing.

It makes it much easier to exchange keys because the keys used in asymmetric encryption are themselves asymmetric.

Now, let's look at exactly what this means.

To use asymmetric encryption, the first stage is for the cap ruler and the robot channel to agree an asymmetric algorithm to use, and then create encryption keys for the algorithm, which logically enough will be asymmetric encryption keys.

Asymmetric encryption keys are formed of two parts, a public key and a private key.

For both sides to be able to send and receive to each other, then both sides would need to make both public and private keys.

To keep the diagram simple, we're going to use the example of where the cap ruler will be sending the battle plans to the robot channel, so only the robot channel in this scenario will need to generate any keys.

Now, a public key can be used to generate ciphertext, which can only be decrypted by the corresponding private key.

The public key cannot decrypt data that it was used to encrypt, only the private key can decrypt that data.

This means the private key needs to be guarded really carefully because it's what's used to decrypt data.

If it leaks, the battle plans could be compromised.

The public key, it's just used to encrypt, and so the robot general uploads his public key to his cloud storage so that anyone can access it.

The worst thing that could happen to anyone who obtains the robot general's public key is that he or she could use it to encrypt plaintext into ciphertext that only the robot general could decrypt.

So there's no downside to anyone getting hold of the robot general's public key.

So with asymmetric encryption, there's no requirement to exchange keys in advance.

As long as the robot general uploaded his public key to somewhere that was accessible to the world, then the first step would be for the cap ruler to download the robot general's public key.

Remember, this isn't sensitive.

Anyone can use it to encrypt data for the robot general, and that's it.

That's the only thing that the public key can do in this scenario.

So using the general's public key and the plaintext battle plans, the asymmetric algorithm would generate some ciphertext.

The ciphertext can then be transmitted to the robot general, and once received, only the robot general could decrypt that data.

This time, though, there's no key exchange required because the rulers are using asymmetric encryption.

The general already has his private key, and so he provides that private key and the ciphertext to the algorithm, which decrypts the ciphertext back into plaintext, and then the robot general has a copy of plaintext battle plans.

Asymmetric encryption is generally used where two or more parties are involved, and generally when those parties have never physically met before.

Issues by PTP, popular email and file encryption system.

Issues by SSL or TLS, which is a system for encrypting browser communications.

And issues by SSH, a popular method to securely access servers using key-based authentication.

Now, asymmetric encryption is computationally much more difficult to do than symmetric, and so many processors use asymmetric encryption to initially agree and communicate symmetric key, and then the symmetric key is used for communication between those two parties from that point onward.

Okay, so this is the end of part one of this lesson.

It was getting a little bit on the long side, and so I wanted to add a break.

It's an opportunity just to take a rest or grab a coffee.

Part two will be continuing immediately from the end of part one.

So go ahead, complete video, and when you're ready, join me in part two.

This is an idea that I found interesting even before taking this class. In the Spanish classes I've taken in high school, the learning process was never enjoyable because the goal of the class was just to do well on the tests. This mentality made me reluctant to speak Spanish outside of a class setting because I feared making a mistake. Even forgetting a conjugation would prevent me from trying at all. However, it's important to realize that even native speakers make mistakes all the time. Many native English speakers have terrible grammar but they are still considered fluent. Worrying about perfection in language is futile since there are also many variations in dialect as well.

I think the fact that people have a tendency to fall into a fixed mindset as they grow older is not because they can't learn new things, but they simply don't see a need to. When a child is learning their first language, it's necessary to function in society, so they have to learn. For an older child or adult who is learning a second language, such as in a classroom, there isn't really a necessity to use it outside of that environment, so when they fail to retain what they've learned, if they see no reason to keep trying then they simply won't.

Reviewer #1 (Public review):

Summary:

In the abstract and throughout the paper, the authors boldly claim that their evidence, from the largest set of data ever collected on inattentional blindness, supports the views that "inattentionally blind participants can successfully report the location, color, and shape of stimuli they deny noticing", "subjects retain awareness of stimuli they fail to report", and "these data...cast doubt on claims that awareness requires attention." If their results were to support these claims, this study would overturn 25+ years of research on inattentional blindness, resolve the rich vs. sparse debate in consciousness research, and critically challenge the current majority view in cognitive science that attention is necessary for awareness.

Unfortunately, these extraordinary claims are not supported by extraordinary (or even moderately convincing) evidence. At best, the results support the more modest conclusion: If sub-optimal methods are used to collect retrospective reports, inattentional blindness rates will be overestimated by up to ~8% (details provided below in comment #1). This evidence-based conclusion means that the phenomenon of inattentional blindness is alive and well as it is even robust to experiments that were specifically aimed at falsifying it. Thankfully, improved methods already exist for correcting the ~8% overestimation of IB rates that this study successfully identified.

Comments:

(1) In experiment 1, data from 374 subjects were included in the analysis. As shown in figure 2b, 267 subjects reported noticing the critical stimulus and 107 subjects reported not noticing it. This translates to a 29% IB rate, if we were to only consider the "did you notice anything unusual Y/N" question. As reported in the results text (and figure 2c), when asked to report the location of the critical stimulus (left/right), 63.6% of the "non-noticer" group answered correctly. In other words, 68 subjects were correct about the location while 39 subjects were incorrect. Importantly, because the location judgment was a 2-alternative-forced-choice, the assumption was that if 50% (or at least not statistically different than 50%) of the subjects answered the location question correctly, everyone was purely guessing. Therefore, we can estimate that ~39 of the subjects who answered correctly were simply guessing (because 39 guessed incorrectly), leaving 29 subjects from the non-noticer group who may have indeed actually seen the location of the stimulus. If these 29 subjects are moved to the noticer group, the corrected rate of IB for experiment 1 is 21% instead of 29%. In other words, relying only on the "Y/N did you notice anything" question leads to an overestimate of IB rates by 8%. This modest level of inaccuracy in estimating IB rates is insufficient for concluding that "subjects retain awareness of stimuli they fail to report", i.e. that inattentional blindness does not exist.

In addition, this 8% inaccuracy in IB rates only considers one side of the story. Given the data reported for experiment 1, one can also calculate the number of subjects who answered "yes, I did notice something unusual" but then reported the incorrect location of the critical stimulus. This turned out to be 8 subjects (or 3% of the "noticer" group). Some would argue that it's reasonable to consider these subjects as inattentionally blind, since they couldn't even report where the critical stimulus they apparently noticed was located. If we move these 8 subjects to the non-noticer group, the 8% overestimation of IB rates is reduced to 6%.

The same exercise can and should be carried out on the other 4 experiments, however, the authors do not report the subject numbers for any of the other experiments, i.e., how many subjects answered Y/N to the noticing question and how many in each group correctly answered the stimulus feature question. From the limited data reported (only total subject numbers and d' values), the effect sizes in experiments 2-5 were all smaller than in experiment 1 (d' for the non-noticer group was lower in all of these follow-up experiments), so it can be safely assumed that the ~6-8% overestimation of IB rates was smaller in these other four experiments. In a revision, the authors should consider reporting these subject numbers for all 5 experiments.

(2) Because classic IB paradigms involve only one critical trial per subject, the authors used a "super subject" approach to estimate sensitivity (d') and response criterion (c) according to signal detection theory (SDT). Some readers may have issues with this super subject approach, but my main concern is with the lack of precision used by the authors when interpreting the results from this super subject analysis.

Only the super subject had above-chance sensitivity (and it was quite modest, with d' values between 0.07 and 0.51), but the authors over-interpret these results as applying to every subject. The methods and analyses cannot determine if any individual subject could report the features above-chance. Therefore, the following list of quotes should be revised for accuracy or removed from the paper as they are misleading and are not supported by the super subject analysis:

"Altogether this approach reveals that subjects can report above-chance the features of stimuli (color, shape, and location) that they had claimed not to notice under traditional yes/no questioning" (p.6)

"In other words, nearly two-thirds of subjects who had just claimed not to have noticed any additional stimulus were then able to correctly report its location." (p.6)

"Even subjects who answer "no" under traditional questioning can still correctly report various features of the stimulus they just reported not having noticed, suggesting that they were at least partially aware of it after all." (p.8)

"Why, if subjects could succeed at our forced-response questions, did they claim not to have noticed anything?" (p.8)

"we found that observers could successfully report a variety of features of unattended stimuli, even when they claimed not to have noticed these stimuli." (p.14)

"our results point to an alternative (and perhaps more straightforward) explanation: that inattentionally blind subjects consciously perceive these stimuli after all... they show sensitivity to IB stimuli because they can see them." (p.16)

"In other words, the inattentionally blind can see after all." (p.17)

(3) In addition to the d' values for the super subject being slightly above zero, the authors attempted an analysis of response bias to further question the existence of IB. By including in some of their experiments critical trials in which no critical stimulus was presented, but asking subjects the standard Y/N IB question anyway, the authors obtained false alarm and correct rejection rates. When these FA/CR rates are taken into account along with hit/miss rates when critical stimuli were presented, the authors could calculate c (response criterion) for the super subject. Here, the authors report that response criteria are biased towards saying "no, I didn't notice anything". However, the validity of applying SDT to classic Y/N IB questioning is questionable.

For example, with the subject numbers provided in Box 1 (the 2x2 table of hits/misses/FA/CR), one can ask, 'how many subjects would have needed to answer "yes, I noticed something unusual" when nothing was presented on the screen in order to obtain a non-biased criterion estimate, i.e., c = 0?' The answer turns out to be 800 subjects (out of the 2761 total subjects in the stimulus-absent condition), or 29% of subjects in this condition.

In the context of these IB paradigms, it is difficult to imagine 29% of subjects claiming to have seen something unusual when nothing was presented. Here, it seems that we may have reached the limits of extending SDT to IB paradigms, which are very different than what SDT was designed for. For example, in classic psychophysical paradigms, the subject is asked to report Y/N as to whether they think a threshold-level stimulus was presented on the screen, i.e., to detect a faint signal in the noise. Subjects complete many trials and know in advance that there will often be stimuli presented and the stimuli will be very difficult to see. In those cases, it seems more reasonable to incorrectly answer "yes" 29% of the time, as you are trying to detect something very subtle that is out there in the world of noise. In IB paradigms, the stimuli are intentionally designed to be highly salient (and unusual), such that with a tiny bit of attention they can be easily seen. When no stimulus is presented and subjects are asked about their own noticing (especially of something unusual), it seems highly unlikely that 29% of them would answer "yes", which is the rate of FAs that would be needed to support the null hypothesis here, i.e., of a non-biased criterion. For these reasons, the analysis of response bias in the current context is questionable and the results claiming to demonstrate a biased criterion do not provide convincing evidence against IB.

(4) One of the strongest pieces of evidence presented in the entire paper is the single data point in Figure 3e showing that in Experiment 3, even the super subject group that rated their non-noticing as "highly confident" had a d' score significantly above zero. Asking for confidence ratings is certainly an improvement over simple Y/N questions about noticing, and if this result were to hold, it could provide a key challenge to IB. However, this result hinges on a single data point, it was not replicated in any of the other 4 experiments, and it can be explained by methodological limitations. I strongly encourage the authors (and other readers) to follow up on this result, in an in-person experiment, with improved questioning procedures.

In the current Experiment 3, the authors asked the standard Y/N IB question, and then asked how confident subjects were in their answer. Asking back-to-back questions, the second one with a scale that pertains to the first one (including a tricky inversion, e.g., "yes, I am confident in my answer of no"), may be asking too much of some subjects, especially subjects paying half-attention in online experiments. This procedure is likely to introduce a sizeable degree of measurement error.

An easy fix in a follow-up study would be to ask subjects to rate their confidence in having noticed something with a single question using an unambiguous scale:

On the last trial, did you notice anything besides the cross?

(1) I am highly confident I didn't notice anything else<br /> (2) I am confident I didn't notice anything else<br /> (3) I am somewhat confident I didn't notice anything else<br /> (4) I am unsure whether I noticed anything else<br /> (5) I am somewhat confident I noticed something else<br /> (6) I am confident I noticed something else<br /> (7) I am highly confident I noticed something else

If we were to re-run this same experiment, in the lab where we can better control the stimuli and the questioning procedure, we would most likely find a d' of zero for subjects who were confident or highly confident (1-2 on the improved scale above) that they didn't notice anything. From there on, the d' values would gradually increase, tracking along with the confidence scale (from 3-7 on the scale). In other words, we would likely find a data pattern similar to that plotted in Figure 3e, but with the first data point on the left moving down to zero d'. In the current online study with the successive (and potentially confusing) retrospective questioning, a handful of subjects could have easily misinterpreted the confidence scale (e.g., inverting the scale) which would lead to a mixture of genuine high-confidence ratings and mistaken ratings, which would result in a super subject d' that falls between zero and the other extreme of the scale (which is exactly what the data in Fig 3e shows).

One way to check on this potential measurement error using the existing dataset would be to conduct additional analyses that incorporate the confidence ratings from the 2AFC location judgment task. For example, were there any subjects who reported being confident or highly confident that they didn't see anything, but then reported being confident or highly confident in judging the location of the thing they didn't see? If so, how many? In other words, how internally (in)consistent were subjects' confidence ratings across the IB and location questions? Such an analysis could help screen-out subjects who made a mistake on the first question and corrected themselves on the second, as well as subjects who weren't reading the questions carefully enough. As far as I could tell, the confidence rating data from the 2AFC location task were not reported anywhere in the main paper or supplement.

(5) In most (if not all) IB experiments in the literature, a partial attention and/or full attention trial (or set of trials) is administered after the critical trial. These control trials are very important for validating IB on the critical trial, as they must show that, when attended, the critical stimuli are very easy to see. If a subject cannot detect the critical stimulus on the control trial, one cannot conclude that they were inattentionally blind on the critical trial, e.g., perhaps the stimulus was just too difficult to see (e.g., too weak, too brief, too far in the periphery, too crowded by distractor stimuli, etc.), or perhaps they weren't paying enough attention overall or failed to follow instructions. In the aggregate data, rates of noticing the stimuli should increase substantially from the critical trial to the control trials. If noticing rates are equivalent on the critical and control trials one cannot conclude that attention was manipulated.

It is puzzling why the authors decided not to include any control trials with partial or full attention in their five experiments, especially given their online data collection procedures where stimulus size, intensity, eccentricity, etc. were uncontrolled and variable across subjects. Including such trials could have actually helped them achieve their goal of challenging the IB hypothesis, e.g., excluding subjects who failed to see the stimulus on the control trials might have reduced the inattentional blindness rates further. This design decision should at least be acknowledged and justified (or noted as a limitation) in a revision of this paper.

(6) In the discussion section, the authors devote a short paragraph to considering an alternative explanation of their non-zero d' results in their super subject analyses: perhaps the critical stimuli were processed unconsciously and left a trace such that when later forced to guess a feature of the stimuli, subjects were able to draw upon this unconscious trace to guide their 2AFC decision. In the subsequent paragraph, the authors relate these results to above-chance forced-choice guessing in blindsight subjects, but reject the analogy based on claims of parsimony.

First, the authors dismiss the comparison of IB and blindsight too quickly. In particular, the results from experiment 3, in which some subjects adamantly (confidently) deny seeing the critical stimulus but guess a feature at above-chance levels (at least at the super subject level and assuming the online subjects interpreted and used the confidence scale correctly), seem highly analogous to blindsight. Importantly, the analogy is strengthened if the subjects who were confident in not seeing anything also reported not being confident in their forced-choice judgments, but as mentioned above this data was not reported.

Second, the authors fail to mention an even more straightforward explanation of these results, which is that ~8% of subjects misinterpreted the "unusual" part of the standard IB question used in experiments 1-3. After all, colored lines and shapes are pretty "usual" for psychology experiments and were present in the distractor stimuli everyone attended to. It seems quite reasonable that some subjects answered this first question, "no, I didn't see anything unusual", but then when told that there was a critical stimulus and asked to judge one of its features, adjusted their response by reconsidering, "oh, ok, if that's the unusual thing you were asking about, of course I saw that extra line flash on the left of the screen". This seems like a more parsimonious alternative compared to either of the two interpretations considered by the authors: (1) IB does not exist, (2) super-subject d' is driven by unconscious processing. Why not also consider: (3) a small percentage of subjects misinterpreted the Y/N question about noticing something unusual. In experiments 4-5, they dropped the term "unusual" but do not analyze whether this made a difference nor do they report enough of the data (subject numbers for the Y/N question and 2AFC) for readers to determine if this helped reduce the ~8% overestimate of IB rates.

(7) The authors use sub-optimal questioning procedures to challenge the existence of the phenomenon this questioning is intended to demonstrate. A more neutral interpretation of this study is that it is a critique on methods in IB research, not a critique on IB as a manipulation or phenomenon. The authors neglect to mention the dozens of modern IB experiments that have improved upon the simple Y/N IB questioning methods. For example, in Michael Cohen's IB experiments (e.g., Cohen et al., 2011; Cohen et al., 2020; Cohen et al., 2021), he uses a carefully crafted set of probing questions to conservatively ensure that subjects who happened to notice the critical stimuli have every possible opportunity to report seeing them. In other experiments (e.g., Hirschhorn et al., 2024; Pitts et al., 2012), researchers not only ask the Y/N question but then follow this up by presenting examples of the critical stimuli so subjects can see exactly what they are being asked about (recognition-style instead of free recall, which is more sensitive). These follow-up questions include foil stimuli that were never presented (similar to the stimulus-absent trials here), and ask for confidence ratings of all stimuli. Conservative, pre-defined exclusion criteria are employed to improve the accuracy of their IB-rate estimates. In these and other studies, researchers are very cautious about trusting what subjects report seeing, and in all cases, still find substantial IB rates, even to highly salient stimuli. The authors should consider at least mentioning these improved methods, and perhaps consider using some of them in their future experiments.

Adam Grant said something along the lines of that your identity is a not what you believe but what you value. That makes a lot of sense. We can have values that influence how we believe things should be done. That doesn’t always mean that they need to be done in that way but it can help us see that deep down our values are what influence those things.

I found it interesting when they talked about how a scientist is trying to find the truth, not spread it. In comparison to a pastor who has the truth and is trying to spread it.

A healthy relationship is when people understand that great minds do NOT think alike. We are taught that great minds think alike but this challenges that thought. I think it’s true, when we are able to disagree we admit that there is more than one way to look at things.

The more we invest in a relationship the more we build it.

If we don’t challenge each other on the small things we will never be able to challenge the big things. It is all about learning how to disagree peacefully. We don’t have to change our minds, but if we are at least open to acknowledging that we may be wrong that will give us more intellectual freedom.

When they discussed that parents who had more arguments raised more creative children I at first was confused. But, with an argument it opens the door to other perspectives and possibilities which would naturally increase the creativity. Again I think it depends on the way we argue, if we are doing it just to spite someone that probably won’t teach creativity or maybe it will. I’m still figuring out what this exactly looks like.

Willpower is motivation not self-control

People will attach to a group. They think that the group is a part of who they are. It is so crazy how attached we can get to groups, we think that they define us but they really don’t.

I really liked when Adam Grant said something along the lines of, “I don’t need you to think I’m right, I just need you to recognize that you might be wrong.” That is such powerful thinking. We don’t always have to be right. Sometimes us recognizing that we might be wrong is the right thing for us to be able to become more open-minded.

I also really liked when Adam Grant said “Character is not how you treat people when things are going your way, it’s how you show up on a hard day.” If everything were sunshine and rainbows in our lives then yes, obviously it’s going to be easier to be kind to someone but when we have a hard day but still choose to be kind that’s what really shows our character.

I am going to ponder the last question they asked, “What do you do when your values are tested?” This is a really good question to ask myself especially before going into our big research paper.

**Kaylie Jensen ** The world is constantly changing and you can either be a part of it or not. You should always be willing to change. There is good change, and bad change, but the reason for change is important. Who you are is not what you believe, but what you value. And we all have values that affect what we do and say. And we all have different views on particular things. Don’t let your ideas become your identity. It’s okay to disagree with people. It’s okay to say what you believe, but don’t do it in a mean way. There is always a right and wrong way to do things. Don’t be afraid to challenge people on the small things, because it leads up to the big things. Learn how to disagree. There isn’t one way to think. Don’t be afraid to question things. We are all a work in progress, we all have to work on ourselves. And if we don’t believe something personally we throw it out. Be open to different opinions. You don’t have to agree on everything, but just remember that they are human beings. The most important connections you will make are in the rifts, where the real friendships are made. True character is what you do when your values are tested.

The concept of "shape completion" mentioned in this section is very interesting, and it emphasizes the importance of machine learning in robot perception. Completing the blind spots of vision through learned experience not only reflects the practical application of artificial intelligence, but also reveals the complexity of robots solving incomplete perception problems in the real world.

I resonate with this idea because, as a teacher, I’ve also found that fostering a sense of community in the classroom is essential. When students feel connected and valued, it creates a space where they are more open to engaging in discussions and pushing their intellectual boundaries. For me, building community isn’t just about creating a safe environment—it’s about cultivating trust and shared commitment to learning, which enhances everyone’s experience and development.

As a Sunday school teacher for kindergarten students, I've seen firsthand how important it is to create an environment where every child feels included and valued. When I teach, I make sure to use stories, examples, and activities that reflect the different backgrounds and experiences of the children in my class. It's not just about teaching religious lessons, but about helping them understand that everyone is unique and important. I've learned that even at a young age, kids notice when they feel left out or different, so it's crucial to make inclusivity a lived experience, not just something we talk about.

Educational inequality has a ripple effect that goes far beyond the classroom, shaping the entire course of a student's life. The fact that success in today’s world is so closely tied to a college degree highlights just how deep this problem runs. It’s frustrating to think that a student's potential is often dictated by the resources their family can provide, rather than their talents or drive. Wealthier students have the advantage of tutors, better schools, extracurricular activities, and financial stability, while students from lower-income families may be just as capable but are held back by factors beyond their control.

I completely agree with this statement. Public education shapes how we think, behave, and engage with society. It’s the foundation of how we learn to interact with others, understand societal norms, and participate in our communities. This role makes education so much more than just academics—it’s a tool for creating engaged, thoughtful citizens. It’s frustrating that this powerful mechanism isn’t used more effectively to promote equity and challenge the structures that perpetuate inequality.

This statistic really makes me think about how investing in early education isn’t just good for children—it’s an investment in society as a whole. The long-term savings from reducing crime and reliance on public assistance are significant. It’s frustrating that more isn’t done to prioritize this, knowing that early education can have such a powerful impact. We’re not just helping individual kids; we’re creating a stronger, more equitable society for everyone.

This sentence really makes me reflect on the ways employers can exploit their workers, especially those in low-wage jobs. When I think about my sister’s experience, it frustrates me that companies can deliberately keep employees just under the threshold for benefits like health insurance. It’s a reminder of how systemic inequality isn’t just about education or wages but also about access to essential services like healthcare. These kinds of practices only deepen the struggles faced by working-class families.

Middle school is such a important time in education, and this passage makes me realize just how early the stratification between students begins. It's alarming to think that by middle school, students are already being sorted into paths that will determine their future access to higher education and social mobility. The lack of funding for public schools, particularly in lower-income areas, means that students in these schools are set up to fall behind their peers in wealthier districts. It’s frustrating to see how much potential is wasted simply because of unequal access to resources, qualified teachers, and support systems. This divide underscores the urgent need for systemic reform to ensure that all students, regardless of background, have an equal shot at success.

I don't think that it's as easy as he says it is. His family had the resources and means to back his business idea up. Many people may not have the same. I think enforcing that they should obtain an education is important, as that can open many opportunities up for growth in the workplace; however, starting a business seems to be more of a luxury that can only start after a couple of years of working for someone else (if you come from a low socioeconomic background).

Welcome back.

In this video, I want to talk in general about application layer firewalls, also known as layer 7 firewalls, named after the layer of the OSI model that they operate at.

Now I want to keep this video pretty generic and talk about how AWS implement this within their product set in a separate video.

So let's jump in and get started.

Now before I talk about the high level architecture and features of layer 7 firewalls, let's quickly refresh our knowledge of layer 3, 4 and 5.

So we start with a layer 3 and 4 firewall, which is helping to secure the Categorum application.

Now this is accessed by millions of people globally because it's that amazing.

Now because this is layer 3 and 4, the firewall sees packets and segments, IP addresses and ports.

It sees two flows of communications, requests from the laptop to the server, and then responses from the server back to the laptop.

Because this firewall is limited to layer 3 and 4 only, these are viewed as separate and unrelated.

You need to think of these as different streams of data, request and response, even though they're part of the same communication from a human perspective.

Now if we enhance the firewall, this time adding session capability, then the same communication between the laptop and server can be viewed as one.

The firewall understands that the request and the response are part of the same session, and this small difference both reduces the admin overhead, so one rule instead of two, but this also lets you implement more contextual security, where you can think of response traffic in the context that it's response to an original request, and treat that differently than traffic in the same direction, which is not a response.

Now this next point is really important.

In both cases, these firewalls don't understand anything above the layer at which they operate.

The top firewall operates layer 3 and 4, so it understands layers 1, 2, 3 and 4.

The bottom firewall does this, plus layer 5.

Now what this means is that both of them can see IP addresses, ports, flags, and the bottom one can do all this, and additionally, it can understand sessions.

Neither of them though can understand the data which flows over the top of this.

They have no visibility into layer 7, for example, HTTP.

So they can't see headers or any of the other data that's been transferred over HTTP.

To them, the layer 7 stuff is opaque.

A cat image is the same as a dog image, is the same as some malware, and this is a significant limitation, and it exposes the things that we're protecting to a wide range of attacks.

Now layer 7 firewalls fix many of these limitations, so let's take a look at how.

Let's consider the same architecture where we have a client on the left, and then a server or application on the right that we're trying to protect.

In the middle we have a layer 7 firewall, and so that you'll remember it's a layer 7 firewall, let's add a robot, a smarter robot.

With this firewall, we still have the same flow of packets and segments, and a layer 7 firewall can understand all of the lower layers, but it adds additional capabilities.

Let's consider this example where the Categorum application is connected using a HTTPS connection.

So encrypted HTTP and HTTP is the layer 7 protocol.

The first important thing to realize is that layer 7 firewalls understand various layer 7 protocols, and the example we're stepping through is HTTP.

So they understand how that protocol transfers data, its architecture, headers, data, hosts, all the things which happen at layer 7 or below.

It also means that it can identify normal or abnormal elements of a layer 7 connection, which means it can protect against various protocol specific attacks or weaknesses.

In this example, so a HTTPS connection to the Categorum server, the HTTPS connection would be terminated on the layer 7 firewall.

So while the client thinks that it's connecting to the server, the HTTPS tunnel would be stripped away, leaving just HTTP, which it could analyze as it transits through the firewall.

So a new HTTPS connection would be created between the layer 7 firewall and the back end server.

So from the server and client's perspective, this process is occurring transparently.

The crucial part of this is that between the original HTTPS connection and the new HTTPS connection, the layer 7 firewall sees an un-imcripted HTTP connection.

So this is plain text, and because the firewall understands the layer 7 protocol, it can see and understand everything about this protocol stream.

Data at layer 7 can be inspected, blocked, replaced, or tagged, and this might be protecting against adult content, spam, off-topic content, or even malware.

So in this example, you might be looking to protect the integrity of the Categorum application.

You'll logically allow cat pictures, but might be less okay with doggoes.

You might draw a line and not allow other animals.

Sheik, for example, might be considered spam.

Maybe you're pretty open and inclusive and only block truly dangerous content such as malware and other exploits.

Because you can see and understand one or more application protocols, you can be very granular in how you allow or block content.

You can even replace content.

So if adult images flow through, these can be replaced with a nice kitten picture or other baby animals.

You can even block specific applications such as Facebook and even block the flow of business data leaving the organization onto services such as Dropbox.

The key thing to understand is that the layer 7 firewall keeps all of the layer 3, 4, and 5 features, but can react to the layer 7 elements.

This includes things like DNS names which are used, the rates of flow for county connections per second, you can even react to content or headers, whatever elements are contained in that specific layer 7 protocol which the firewall understands.

Now some layer 7 firewalls only understand HTTP, some understand SMTP which is the protocol used for email delivery.

The limit is only based on what the firewall software supports.

Now that's everything that I wanted to cover at a high level.

Coming up in future videos, I'm going to be covering how AWS implements layer 7 firewall capability into its product set.

For now though, this high level understanding is what I wanted to help with in this video.

So go ahead and complete the video.

Thanks for watching.

I'm already, I look forward to you joining me in the next.

Welcome back and in this lesson I want to talk in a little bit of detail about fiber optic cables.

If you're involved with networking in any way, then you need to be comfortable with how they work, their characteristics and the differences between the various types.

Now this matters for the real world and if you need to work with any physical networking services, including AWS Direct Connect.

Now let's just jump in and get started.

Fiber optic cables are an alternative way to transmit data versus copper cables.

Where copper cables use changes in electrical signals to transmit data over a copper medium, fiber optic cables use a thin glass or plastic core surrounded by various protective layers.

The core is about the diameter of a human hair.

The cable that you can see and touch is that core surrounded by a lot of protection.

If you just handle the core on its own, it would be pretty susceptible to damage.

Now fiber optic cables, as the name suggests, transmit light over the glass or plastic medium, so light over glass or plastic versus electrical signals over copper.

These unique elements mean that the cable can cover much larger distances and achieve much higher speeds versus copper.

At the time of creating this lesson, this can be in the regions of terabits per second.

Now fiber is also resistant to electromagnetic interference known as EMI and it's less prone to being impacted by water ingress into a space where the cables are being used.

In general, fiber offers a more consistent experience versus copper cable and so in modern networks, specifically those which require higher speeds and or larger distances, fiber is often preferred versus copper.

You're going to see over time fiber will gradually overtake copper in almost all wired networking situations as it becomes cheaper and easier to install.

It's already used for many global networking, metro networking and even many local area networking applications.

So that's important to understand.

It's going to be used more and more in the future.

Now in terms of physical makeup, this is what a typical fiber cable looks like externally.

There are two different things that you need to think about and this is common with any form of networking cable or any form of cable in general.

There's the cable choice which will influence the physical characteristics so how fast data can be transferred and over what distances.

Then you have the cable connectors and these generally affect what the cable can be connected to so linked to physical ports on the networking equipment but they can also influence some of the physical characteristics in terms of distance ability and speeds.

Now I'm not going to detail all the different fiber cable types in this lesson.

Instead I've included a link attached to this lesson which gives you a good overview of the common cable and connector types within the industry.

Now I want to spend a few minutes talking about the physical construction of fiber cables.

I've talked about how the connectors are different but it's important that you understand the different physical makeups of the cable itself.

Now when we're talking about fiber cable, you'll see it referred to using an X/Y notation.

For example, 9/125.

This defines two parts of the cable.

The first part is the diameter of the core in microns and the second part is the diameter of the cladding.

Now the first bit that surrounds the core.

Both of these are in microns and there are a thousand microns in a millimeter.

Now let's talk about the different components of a fiber cable and I'm mainly going to be covering the fiber core, the fiber cladding and the buffer.

And don't worry, the functions of each of these will make sense in a second.

Now we're going to start with the core and this is the part of the cable where the light is carried which allows for the transfer of data.

This part on a 9/125 cable is tinier.

It's only 9 microns across.

So if you look at the right of the screen, you have the core, then you have the transmitter receive optics at each side and the light flows through the core along the length of the cable.

I'll talk more about this in a moment but the light doesn't flow in a straight line.

Now we're bouncing off the inside edges of the core which is why the size of the core matters.

Now surrounding the core is the cladding and this is a material which has a lower refractive index versus the core.

And this means that it acts as a container to keep the light bouncing around inside the core.

Different types of fiber have different sizes of core and cladding and both of them radically impact the physical characteristics of the cable.

And this is where we move on to the less important parts of the cable.

The core and cladding were directly responsible for the physical transmission of data but now we're moving on to the protective balance.

So next we have the buffer and the buffer is the thing which adds strength to the cable.

The core and cladding are generally really good at helping to carry data but really bad at withstanding any physical shocks.

The buffer is a combination of coating and strengthening materials such as fibers made out of other materials.

Now don't confuse this type of fiber with fiber optic.

This is just a length of material which is designed to absorb shocks and give physical protection.

And this buffer is surrounded by the cable jacket which is the physical thing that you see when looking at the fiber cable.

It generally has a defined color such as green, orange or blue and this generally gives some indication on the overall capabilities of the cable.

Now I've included a link attached to this lesson which details the common colors and what they mean in terms of the fiber optic cable capabilities.

Now one more thing that I want to cover before we finish this lesson and that's the difference between single mode and multi mode fiber.

The difference might seem pretty nuanced but it's really important to understand.

Let's start with single mode.

Single mode generally has a really small core and it's often 8 to 9 microns in size.

And it generally uses a yellow jacket but this isn't always the case.

Now because of this tiny core, light generally follows a fairly single and straight path down at the core.

There's generally very little bounce and so very little distortion.

Single mode fiber generally uses lasers and so it's more expensive in terms of the optics versus multi mode.

Single mode because of this lack of distortion is great for long distances and it can achieve excellent speeds over these long distances.

Now it's not the fastest type of fiber cable but if you need a combination of high speeds and long distances then it's by far the best.

Single mode fiber can reach kilometers and can do really high speeds at those distances.

Generally in production usage this is 10 gig and above.

Single mode fiber cable itself is generally cheaper than multi mode fiber but the transceivers are things which send and receive light are more expensive versus multi mode.

But this is changing over time and this will probably mean more and more single mode usage within most business applications.

Now multi mode cable generally has a much bigger core and often uses either an orange and aqua or other coloured jacket.

The bigger core means that it can be used with a wider range of wavelengths of light generally at the same time.

For simplicity think of this as different colours of light travelling down the same fiber cable so these different colours can be sent at the same time and don't interfere with each other.

More light means more data so multi mode tends to be faster.

But that comes with a trade off because this leads to more distortion over the light over longer distances.

For that reason multi mode historically has been used for shorter cable runs where speed and cost effectiveness is required.

Now multi mode generally has cheaper LED based optics rather than the more expensive laser optics used within single mode fiber.

Multi mode fiber cable will generally use the prefix OM so OM2, OM3, OM4 and so on each improving the previous ones capabilities.

And multi mode as I mentioned before has a larger core.

At a high level the type of cable you decide on is determined by the distances that you need to transmit data and the speed.

So single mode is just more sturdy, there's less distortion and it can do better speeds over higher distances.

And as the optics prices come down I suspect more people will use single mode even for shorter distances.

Now one final thing I want to cover before we finish up with this lesson and that's fiber optic transceivers.

Now these are generally the things which you plug into networking equipment which allows the networking equipment to connect to fiber optic cables.

They're known as SFP transceiver modules also known as SFP or mini gibix and this stands for single form factor pluggable.

Now these are the things which generate and send or receives light to and from the fiber optic cable.

So these plug into networking equipment, these have optics inside which generate the light or can detect the light and these are used to translate from data to light and from lights to data that networking equipment can use.

Now these transceivers are either multi mode or single mode and they're optimised for a specific cable type.

So you generally buy a transceiver that's designed to be used with a certain type of cable and the transceivers will need to be the same type on both sides or both ends of the fiber optic cable.

Now when you're talking about the connector type and the cable you're generally going to see terms such as 1000 base LX, 10G base LR or 100G base LR4.

And these are often specified by vendors such as AWS to give you an idea on the type of cable and the connector that you need to use to plug into their equipment.

So in the case of AWS DirectConnect the supported types are 1000 base LX, 10G base LR and 100G base LR4.

Now at this point that's everything I wanted to cover at a high level about fiber optic cables and transceivers and once again I've included some links attached to this lesson which go into a little bit more detail if you're interested.

At this point that's everything I wanted to cover to go ahead and complete the video and when you're ready I'll look forward to you joining me in the next.

Welcome back.

In this lesson, I want to cover IPsec fundamentals.

So I want to talk about what IPsec is, why it matters, and how IPsec works at a fundamental level.

Now we have a lot of theory to cover, so let's jump in and get started.

At a foundational level, IPsec is a group of protocols which work together.

Their aim is to set up secure networking tunnels across insecure networks.

For example, connecting two secure networks or more specifically, their routers, called peers, across the public internet.

Now you might use this if you're a business with multiple sites, spread around geographically and want to connect them together, or if you have infrastructure in AWS or another cloud platform and want to connect to that infrastructure.

IPsec provides authentication so that only peers which are known to each other and can authenticate with each other can connect.

And any traffic which is carried by the IPsec protocols is encrypted, which means to one look at secure data which is being carried is ciphertext.

It can't be viewed and it can't be altered without being detected.

Now architecturally, it looks like this.

We have the public internet which is an insecure network full of goblins looking to steal your data.

Over this insecure network, we create IPsec tunnels between peers.

Now these tunnels exist as they're required.

Within IPsec VPNs, there's the concept of interesting traffic.

Now interesting traffic is simply traffic which matches certain rules, and these could be based on network prefixes or match more complex traffic types.

Regardless of the rules, if data matches any of those rules, it's classified as interesting traffic, and the VPN tunnel is created to carry traffic through to its destination.

Now if there's no interesting traffic, then tunnels are eventually torn down only to be reestablished when the system next detects interesting traffic.

The key thing to understand is that even though those tunnels use the public internet for transit, any data within the tunnels is encrypted while transiting over that insecure network.

It's protected.

Now to understand the nuance of what IPsec does, we need to refresh a few key pieces of knowledge.

In my fundamental section, I talked about the different types of encryption.

I mentioned symmetric and asymmetric encryption.

Now symmetric encryption is fast.

It's generally really easy to perform on any modern CPU and it has pretty low overhead.

But exchanging keys is a challenge.

The same keys are used to encrypt and decrypt.

So how can you get the key from one entity to another securely?

Do you transmit it in advance over a different medium, or do you encrypt it?

If so, you run into a catch-22 situation.

How do you securely transmit the encrypted key?

That's why asymmetric encryption is really valuable.

Now it's slower, so we don't want to be using it all the time, but it makes exchanging keys really simple because different keys are used for encryption and decryption.

Now a public key is used to encrypt data, and only the corresponding private key can decrypt that data.

And this means that you can safely exchange the public key while keeping the private key private.

So the aim of most protocols which handle the encryption of data over the internet is to start with asymmetric encryption, use this to securely exchange symmetric keys, and then use those for ongoing encryption.

Now I mentioned that because it will help you understand exactly how IPsec VPN works.

So let's go through it.

IPsec has two main phases.

If you work with VPNs, you're going to hear a lot of talk about phase one or phase two.

It's going to make sense why these are needed by the end of this lesson, but understand that there are two phases in setting up a given VPN connection.

The first is known as Ike phase one.

Ike or Internet Key Exchange, as the name suggests, is a protocol for how keys are exchanged in this context within a VPN.

There are two versions, Ike version one and Ike version two.

Version one logically is older, version two is newer and comes with more features.

Now you don't need to know all the detail right now, just understand that the protocol is about exchanging keys.

Ike phase one is the slow and heavy part of the process.

It's where you initially authenticate using a pre-shared key, so a password of sorts or a certificate.

It's where asymmetric encryption is used to agree on, create and share symmetric keys which are used in phase two.

The end of this phase is what's known as an Ike phase one tunnel or a security association known as an SA.

There's lots of jargon being thrown around, and I'll be showing you how this all works visually in just a moment.

But at the end of phase one, you have a phase one tunnel, and the heavy work of moving towards symmetric keys which can be used for encryption has been completed.

The next step is Ike phase two which is faster and much more agile because much of the heavy lifting has been done in phase one.

Technically, the phase one keys are used as a starting point for phase two.

Phase two is built on top of phase one and is concerned with agreeing encryption methods and the keys used for the bulk transfer of data.

The end result is an IPsec security association, a phase two tunnel which runs over phase one.

Now the reason why these different phases are split up is that it's possible for phase one to be established, then a phase two tunnel created, used and then torn down when no more interesting traffic occurs, but the phase one tunnel stays.

It means that establishing a new phase two tunnel is much faster and less work.

It's an elegant and well designed architecture, so let's look at how this all works together visually.

So this is Ike phase one.

The architecture is a simple one.

Two business sites, site one on the left with the user Bob and site two on the right with the user Julie, and in the middle, the public internet.

The very first step of this process is that the routers, the two peers at either side of this architecture need to authenticate.

Essentially prove their identity which is done either using certificates or pre-shared keys.

Now it's important to understand that this isn't yet about encryption, it's about proving identity.

Proving that both sides agree that the other side should be part of this VPN.

No keys are exchanged, it's just about identity.

Once the identity has been confirmed, then we move on to the next stage of Ike phase one.

In this stage we use a process called Diffie-Hellman Key Exchange.

Now again, I'm sorry about the jargon, but try your best to remember Diffie-Hellman known as DH.

What happens is that each side creates a Diffie-Hellman private key.

This key is used to decrypt data and to sign things.

You should remember this from the encryption fundamentals lesson.

In addition, each side uses that private key and derives a corresponding public key.

Now the public key can be used to encrypt data that only that private key can decrypt.

So at this point, each side has a private key as well as a corresponding public key.

At this point, these public keys are exchanged.

So Bob has Julie's public key and Julie has Bob's public key.

Remember, these public keys are not sensitive and can only be used normally to encrypt data for decryption by the corresponding private key.

The next stage of the process is actually really complicated mathematics, but at a fundamental level, each side takes its own private key and the public key at the other side and uses this to derive what's known as the Diffie-Hellman key.

This key is the same at both sides, but it's been independently generated.

Now again, the maths is something that's well beyond this lesson, but it's at the core of how this phase of VPN works.

And at this point, it's used to exchange other key material and agreements.

This part you can think of as a negotiation.

The result is that each side again independently uses this DH key plus the exchanged key material to generate a final phase one symmetrical key.

This key is what's used to encrypt anything passing through a phase one tunnel known as the Ike Security Association.

Now, if that process seems slow and heavy, that's because it is.

It's both complex and in some ways simplistically elegant at the same time, but it means that both sides have the same symmetric key without that ever having to be passed between them.

And the phase ends with this security association in place and this can be used at phase two.

So let's talk about that next.

So in phase two, we have a few things.

First, a DH key on both sides and the same phase one symmetric key also on both sides.

And then finally, the established phase one tunnel.

During this phase, both of the peers are wanting to agree how the VPN itself will be constructed.

The previous phase was about allowing this exchanging keys and allowing the peers to communicate.

This phase, so Ike phase two, is about getting the VPN up and running, being in a position to encrypt data.

So agreeing how, when and what.

So the first part of this is that the symmetric key is used to encrypt and decrypt agreements and pass more key material between the peers.

The idea is that one peer is informing the other about the range of cybersuits that it supports, basically encryption methods which it can perform.

The other peer, in this example, the right one, will then pick the best shared one.

So the best method which it also supports and it will let the left peer know.

And this becomes the agreed method of communication.

Next, the DH key and the key material exchanged above is used to create a new key, a symmetrical IP set key.

This is a key which is designed for large-scale data transfer.

It's an efficient and secure algorithm.

And the specific one is based on the negotiation which happened above in steps one and two at this phase.

So it's this key which is used for the encryption and decryption of interesting traffic across the VPN tunnel.

Across each phase one tunnel, you actually have a pair of security associations.

One from right to left and one from left to right.

And these are the security associations which are used to transfer the data between networks at either side of a VPN.

Now there are actually two different types of VPN which you need to understand.

Policy-based VPNs and route-based VPNs.

The difference is how they match interesting traffic.

Remember, this is the traffic which gets sent over a VPN.

So with policy-based VPNs, there are rules created which match traffic.

And based on this rule, traffic is sent over a pair of security associations.

One which is used for each direction of traffic.

It means that you can have different rules for different types of traffic.

Something which is great for more rigorous security environments.

Now the other type of VPN are route-based VPNs.

And these do target matching based on prefix.

For example, send traffic for 192.168.0.0/24 over this VPN.

With this type of VPN, you have a single pair of security associations for each network prefix.

This means all traffic types between those networks use the same pair of security associations.

Now this provides less functionality which is much simpler to set up.

To illustrate the differences between route-based and policy-based VPNs, it's probably worth looking visually at the phase one and phase two architectures.

Let's start with a simple route-based VPN.

The phase one tunnel is established using a phase one tunnel key.

Now assuming that we're using a route-based VPN, then a single pair of security associations is created.

One in each direction using a single IPsec key.

So this means that we have a pair of security associations, essentially a single phase two tunnel, running over the phase one tunnel.

That phase two or IPsec tunnel, which is how we talk about the pair of security associations, can be dropped when there is no more interesting traffic and recreated again on top of the same phase one tunnel when new traffic is detected.

But the key thing to understand is that there's one phase one tunnel running one phase two tunnel based on routes.

Running a policy-based VPN is different.

We still have the same phase one tunnel, but over the top of this, each policy match uses an SA pair with a unique IPsec key.

And this allows us to have for the same network different security settings for different types of traffic.

In this example, infrastructure at the top, CCTV in the middle and financial systems at the bottom.

So policy-based VPNs are more difficult to configure, but do provide much more flexibility when it comes to using different security settings for different types of traffic.

Now that, at a very high level, is how VPNs functions, so the security architecture, how everything interacts with everything else.

But for now, that's everything that I wanted to cover.

So go ahead and complete this video, and then when you're ready, I look forward to you joining me in the next.

Welcome back in this video, I want to cover the differences between stateful and stateless firewalls.

And to do that, I need to refresh your knowledge of how TCP and IP function.

So let's just jump in and get started.

In the networking fundamentals videos, I talk about how TCP and IP worked together.

You might already know this if you have networking experience in the real world, but when you make a connection using TCP, what's actually happening is that each side is sending IP packets to each other.

These IP packets have a source and destination IP, and are carried across local networks and the public internet.

Now TCP is a layer 4 protocol which runs on top of IP.

It adds error correction together with the idea of ports.

So HTTP runs on TCP port 80 and HTTPS runs on TCP port 443 and so on.

So keep that in mind as we continue talking about the state of connections.

So let's say that we have a user here on the left, Bob, and he's connecting to the Categoram application running on a server on the right.

What most people imagine in this scenario is a single connection between Bob's laptop and the server.

So Bob's connecting to TCP port 443 on the server, and in doing so, he gets information back.

In this case, many different cat images.

Now you know that below the surface of layer 3, this single connection is handled by exchanging packets between the source and the destination.

Conceptually though, you can imagine that each connection, in this case, is an outgoing connection from Bob's laptop to the server.

Each one of these is actually made up of two different parts.

First, we've got the request part where the client requests some information from the server, in this case on cat images, and then we have the response part where that data is returned to the client.

Now these are both parts of the same interaction between the client and server, but strictly speaking, you can think of these as two different components.

What actually happens as part of this connection setup is this.

First, the client picks a temporary port, and this is known as an ephemeral port.

Now typically this port has a value between 1024 and 65535, but this range is dependent on the operating system which Bob's laptop is using.

Then once this ephemeral port is chosen, the client initiates a connection to the server using a well-known port number.

Now a well-known port number is a port number which is typically associated with one specific popular application or protocol.

In this case, TCP443 is HTTPS.

So this is the request part of the connection.

It's a stream of data to the server.

You're asking for something, some cat pictures or a web page.

Next, the server responds back with the actual data.

The server connects back to the source IP of the request part, in this case Bob's laptop, and it connects to the source port of the request part, which is the ephemeral port which Bob's laptop has chosen.

This part is known as the response.

So the request is from Bob's laptop using an ephemeral port to a server using a well-known port.

The response is from the server on that well-known port, but Bob's laptop on the ephemeral port.

Now it's these values which uniquely identify a single connection.

So that's a source port and source IP, and a destination IP, and a destination port.

Now I hope that this makes sense so far.

If not, then you need to repeat this first part of the video again, because this is really important to understand.

If it does make sense, then let's carry on.

Now let's look at this example in a little bit more detail.

This is the same connection that we looked at on the previous screen.

We have Bob's laptop on the left and a Caterpillar on the right.

Obviously the left is the client and the right is the server.

I also introduced the correct terms on the previous screen, so request and response.

So the first part is the client talking to the server, asking for something, and that's the request, and the second part is the server responding, and that's the response.

But what I want to get you used to is that the directionality depends on your perspective, and let me explain what I mean.

So in this case, the client initiates the request, and I've added the IP addresses on here for both the client and the server.

So what this means is the packets will be sent from the client to the server, and these will be flowing from left to right.

These packets are going to have a source IP address of 119.18.36.73, which is the IP address of the client, so Bob's laptop, and they will have a destination IP of 1.3.3.7, which is the IP address of the server.

Now the source port will be a temporary or ephemeral port chosen by the client, and the destination port will be a well-known port.

In this case, we're using HTTPS, so TCP port 443.

Now if I challenge you to take a quick guess, would you say that this request is outbound or inbound?

If you had to pick, if you had to define a firewall rule right now, would you pick inbound or outbound?

Well, this is actually a trick question, because it's both.

From the client perspective, this request is an outbound connection.

So if you're adding a firewall rule on the client, you would be looking to allow or deny an outbound connection.

From the server perspective, though, it's an inbound connection, so you have to think about perspective when you're working with firewalls.

But then we have the response part from the server through to the client.

This will also be a collection of packets moving from right to left.

This time, the source IP on those packets will be 1.3.3.7, which is the IP address of the server.

The destination IP will be 119.18.36.73, which is the IP address of the client, so Bob's Laptop.

The source port will be TCP port 443, which is the well-known port of HTTPS, and the destination port will be the ephemeral port chosen originally by the client.

Now again, I want you to think about the directionality of this component of the communication.

Is it outbound or inbound?

Well, again, it depends on perspective.

The server sees it as an outbound connection from the server to the client, and the client sees it as an inbound connection from the server to itself.

Now, this is really important because there are two things to think about when dealing with firewall rules.

The first is that each connection between a client and a server has two components, the request and the response.

So the request is from a client to a server, and the response is from a server to a client.

The response is always the inverse direction to the request.

But the direction of the request isn't always outbound and isn't always inbound.

It depends on what that data is together with your perspective.

And that's what I want to talk about a bit more on the next screen.

Let's look at this more complex example.

We still have Bob and his laptop and the Catergram server, but now we have a software update server on the bottom left.

Now, the Catergram server is inside a subnet which is protected by a firewall, and specifically, this is a stateless firewall.

A stateless firewall means that it doesn't understand the state of connections.

What this means is that it sees the request connection from Bob's laptop to Catergram, and the response of from a Catergram to Bob's laptop as two individual parts.

You need to think about allowing or denying them as two parts.

You need two rules.

In this case, one inbound rule which is the request and one outbound rule for the response.

This is obviously more management overhead.

Two rules needed for each thing.

Each thing which you as a human see as one connection.

But it gets slightly more confusing than that.

For connections to the Catergram server, so for example, when Bob's laptop is making a request, then that request is inbound to the Catergram server.

The response, logically enough, is outbound, sending data back to Bob's laptop, but it's possible to have the inverse.

Consider the situation where the Catergram server is performing software updates.

Well, in this situation, the request will be from the Catergram server to the software update server, so outbound, and the response will be from the software update server to the Catergram server, so this is inbound.

So when you're thinking about this, start with the request.

Is the request coming to you or going to somewhere else?

The response will always be in the reverse direction.

So this situation also requires two firewall rules.

One outbound for the request and one inbound for the response.

Now, there are two really important points I want to make about stateless firewalls.

First, for any servers where they accept connections and where they initiate connections, and this is common with web servers which need to accept connections from clients, but where they also need to do software updates.

In this situation, you'll have to deal with two rules for each of these, and they will need to be the inverse of each other.

So get used to thinking that outbound rules can be both the request and the response, and inbound rules can also be the request and the response.

It's initially confusing, but just remember, start by determining the direction of the request, and then always keep in mind that with stateless firewalls, you're going to need an inverse rule for the response.

Now, the second important thing is that the request component is always going to be to a well-known port.

If you're managing the firewall for the Catergram application, you'll need to allow connections to TCP 443.

The response, though, is always from the server to a client, but this always uses a random ephemeral port, because the firewall is stateless, it has no way of knowing which specific port is used for the response, so you'll often have to allow the full range of ephemeral ports to any destination.

This makes security engineers uneasy, which is why stateful firewalls, which I'll be talking about next, are much better.

Just focus on these two key elements that every connection has a request and a response, and together with those, keep in mind the fact that they can both be in either direction, so a request can be inbound or outbound, and a response will always be the inverse to the directionality of the request.

Also, you'll need to keep in mind that any rules that you create for the response will need to often allow the full range of ephemeral ports.

That's not a problem with stateful firewalls, which I want to cover next.

So we're going to use the same architecture.

We've got Bob's laptop on the top left, the Catergram server on the middle right, and the Software Update server on the bottom left.

A stateful firewall is intelligent enough to identify the response for a given request, since the ports and IPs are the same, it can link one to the other, and this means that for a specific request to Catergram from Bob's laptop to the server, the firewall automatically knows which data is the response, and the same is true for software updates.

For a given connection to a software update server, the request, the firewall is smart enough to be able to see the response or the return data from the software update server back to the Catergram server, and this means that with a stateful firewall, you'll generally only have to allow the request or not, and the response will be allowed or not automatically.

This significantly reduces the admin overhead and the chance for mistakes, because you just have to think in terms of the directionality and the IPs and ports of the request, and it handles everything else.

In addition, you don't need to allow the full ephemeral port range, because the firewall can identify which port is being used, and implicitly allow it, based on it being the response to a request that you allow.

Okay, so that's how stateful and stateful firewalls work, and I know it's been a little bit abstract, but this has been intentional, because I want you to understand how they work, and sexually, before I go into more detail with regards to how AWS implements both of these different security firewall standards.

Now, at this point, I've finished with the abstract descriptions, so go ahead and finish this video, and when you're ready, I'll look forward to you joining me in the next.

Education really is the backbone of national development. When we set high standards and create a culture of achievement, we can change not just individual lives but also shape the future of our society. It’s exciting to think about how making quality education accessible to every child can help them reach their full potential, no matter where they come from. By aiming high and building a supportive environment, we can nurture a generation that’s not only ready for academic challenges but also has the skills and confidence to make a positive impact in their communities.

From this statement, it definitely depends on the resources given to students, and that can predict their success. For those who don’t have access to these resources, it’s clear they might not do as well academically compared to those who do. In one of my education classes, we talked about how smaller classrooms with smaller student-to-teacher ratio can positively impact students' learning. Even now in college, when you have a class with 200 students to one professor, it’s so hard to actually engage with the professor or for them to even know who you are and understand your academic goals. This definitely emphasizes that can even come out to be the ratio of students compared to teachers.

It’s fascinating to think about how Louis XIV ruled for 72 years, especially since it’s so different from what we see today, where leaders don’t typically stay in power for that long. Starting at just 4 years old and still managing to strengthen France and the monarchy by the 1680s really shows how significant his reign was.

Quote: "Ultimately, however, the role of sanctions and export controls now is to change the structure of Washington and its allies’ economic relationship with Russia, ensuring that whatever trade remains benefits the United States and Europe more than it benefits the Kremlin."

Thoughts: I think in this part of the article the author is trying to finalize his piece by bringing us into his claim that the role of the counteractions done by the U.S. and EU on Russia is not neccesarily to escalate, deescalate, or make themselves stronger. It is just to make sure that when all the dust settles and all damage is done to both parties. Russia has suffered more relative to it's original place than the U.S. and EU have suffered relative to their original place. It's like a petty children's fight where they care more about how much they hurt the other one than how much they lost.

"Sanctions and export controls can limit Russia’s ability to produce and develop advanced military equipment."

Annotation #3: I never thought about how these sanctions can actually slow down Russia’s military development. It’s interesting because I always assumed sanctions just hurt the economy, but this could really change their ability to build weapons too. This gave me a new perspective on how economic moves like this can affect global military balance. It makes me think that over time, Russia’s military power could shrink, which ties back to our question about the long-term global effects.

Tiresias, who is supposed to transcend genders and signify absolute wisdom, still falls victim to a patriarchal order. In both The Metamorphoses of Ovid and Lempriere’s description, while Tiresias is both male and female, the readers get a sense that female is only his secondary sex. Despite his continuous switching between the two genders, the pronoun used for Tiresias never changes – it remains a “he.” In The Metamorphose of Ovid, even after Tiresias becomes a woman, the author states that “he” comes “upon the serpents.” Similarly, Lempriere describes Tiresias’s transformation as “he himself suddenly changed into a girl.” The use of both “he” and “himself” creates particular emphasis on his masculinity as his primary identity. It is further described as an “original sex,” suggesting an inherently secondary role of a female identity. Societal expectations force him into a binary framework and prioritize masculinity, reducing femininity to a secondary status.

Eliot picks up on this binary notion and further builds off of it. He describes Tiresias as an “old man with wrinkled female breasts.” Tiresias’s male identity is absolute – he is a “man,” while his femininity becomes reduced to his “wrinkled female breasts” – a mere physical attribute. The use of “wrinkled” invokes a sense of a decay of and disdain for Tiresias’s femininity. While it does directly relate to aging and a sense of decay, wrinkles often appear by attempting to shrink or contract a substance. A symbol of Tiresias’s female identity, thus, becomes shrunken and diminished. Additionally, since wrinkles is a product of an external perception, the readers gain a sense that it is the patriarchal narrative that insists on prioritizing his masculine side. This description suggests that feminine identity is viewed as less valuable, condensed or even grotesque within the patriarchal lens.

Furthermore, the shift from the more neutral term "breasts" to the more vulgar "dugs" in Eliot’s portrayal implies a degradation of femininity. According to Collins Dictionary, "dug" can be “used, vulgarly or contemptuously, of a woman's breast,” which reinforces the idea that Tiresias's feminine side is not only secondary but also viewed with disdain. Does Teresias feel a sense of contempt for his feminine side? Or, more likely, is it the external perception that belittles and disregards this femininity? In any case, Eliot uses this transition in terminology to underscore the tension between Tiresias’s dual identities.

Eliot describes the existence of female identity within a patriarchal framework more directly when he describes a sexual assault of a female character, who, following the intercourse, thinks, “Well now that’s done and I am glad it’s over.” She resigns to these circumstances and the external influence makes her disregard her own agency. Just as Tiresias’s female identity becomes secondary and almost grotesque, the female character becomes a mere vessel of satisfaction for her partner. As a result of her submissiveness, her thoughts become detached. She is “hardly aware” of her lover’s departure, her thoughts are “half-formed” and her hand is “automatic.” These descriptions invoke an image of a dehumanized individual: she is stripped from the full consciousness and self-awareness of a human; instead, her movements and thoughts remind that of a lifeless robot. Within this binary system, both the feminine aspect of Tiresias and the female character's humanity are diminished under the weight of patriarchal dominance. Femininity and female identity become dismissed, dehumanized, and reduced to a mere tool for male satisfaction.

Algorithmic policy for memory allocation. Checks if there are enough free pages for an allocation request based on a watermark-based allocation policy criteria. Is is tone by checking the number of free pages in the zone to a threshold "mark".

It is unfortunate that the act of an individual person can permanently taint the work of hundreds, directly affecting those around him who weren't even involved in the perpetrated crime. It is also unfortunate that in the process of trying to protect women, or any victims for a matter of fact, we unintentionally are harming them as well. In this case, it seems that the choice to pull the reruns of The Cosby Show was more of a publicity stunt instead of a legitimate attempt to protect the demographic most harmed by Bill Cosby's actions. They could have easily simply done something to his residual payments to prevent him from profiting off the work he worked in—not his work, he was simply just one of the many people that helped The Cosby Show become reality. This is why it's important to think thoroughly of the consequences an action may have on not just the perpetuator, but also the victims and other parties, directly or indirectly, involved.

I agree with this statement. The actions of the artist does not discredit the work that they have done, but they should still, and always be, held accountable for the harm they have done to others. Given the speed at which news changes nowadays, it’s common for individuals’ crimes to be overlooked and for them to regain their previous reputations. Violence is not excusable, no matter the individual and their identity, and it is important that we do not allow such actions to be 'normalized within the industry,' and if possible, not support such individuals directly.

The harm that has been done upon others, especially marginalized groups, is often overlooked if the products created from that harm produces profit. This is why it's important not to diminish anyone's story no matter how popular or reputable the other party is. This paragraph's last line also comments on the importance of power in groups, as the stories of other people allows for those with similar stories to be more confident and come out as well.

The impact of language, especially in news articles, is blatantly clear. We get a lot of our information of the outside world from the news, and the first source of information we see and consume from these news articles are the headlines. These titles often introduce bias, either exaggerating or downplaying the content to attract readers and generate revenue, which is why it's important to also read the content of the article, and other articles, to fully grasp the situation the article is reporting on.

This is particularly problematic in cases of sexual violence, where articles frequently minimize the severity of the crime and may even favor the abuser. The distinction between "sex" and "sexual violence" hinges on one important element: consent. This difference is significant. It's not uncommon for me to see articles that cover rape, not label what is rape as rape (a 'recent' case I could think of is the mass rape trial in France). The connotations of the words used shape our perceptions (e.g. words like dislike, hate, detest, loathe---we feel different things regarding each of these words even though they are often referred to as synonym of each other), influencing how we judge the seriousness of these incidents.

Author response:

The following is the authors’ response to the original reviews.

Public Reviews:

Reviewer #1 (Public Review):

Summary:

The overall analysis and discovery of the common motif are important and exciting. Very few human/primate ribozymes have been published and this manuscript presents a relatively detailed analysis of two of them. The minimized domains appear to be some of the smallest known self-cleaving ribozymes.

Strengths:

The manuscript is rooted in deep mutational analysis of the OR4K15 and LINE1 and subsequently in modeling of a huge active site based on the closely-related core of the TS ribozyme. The experiments support the HTS findings and provide convincing evidence that the ribozymes are structurally related to the core of the TS ribozyme, which has not been found in primates prior to this work.

Weaknesses:

(1) Given that these two ribozymes have not been described outside of a single figure in a Science Supplement, it is important to show their locations in the human genome, present their sequence and structure conservation among various species, particularly primates, and test and discuss the activity of variants found in non-human organisms. Furthermore, OR4K15 exists in three copies on three separate chromosomes in the human genome, with slight variations in the ribozyme sequence. All three of these variants should be tested experimentally and their activity should be presented. A similar analysis should be presented for the naturally-occurring variants of the LINE1 ribozyme. These data are a rich source for comparison with the deep mutagenesis presented here. Inserting a figure (1) that would show the genomic locations, directions, and conservation of these ribozymes and discussing them in light of this new presentation would greatly improve the manuscript. As for the biological roles of known self-cleaving ribozymes in humans, there is a bioRxiv manuscript on the role of the CPEB3 ribozyme in mammalian memory formation (doi.org/10.1101/2023.06.07.543953), and an analysis of the CPEB3 functional conservation throughout mammals (Bendixsen et al. MBE 2021). Furthermore, the authors missed two papers that presented the discovery of human hammerhead ribozymes that reside in introns (by de la PeÃ{plus minus}a and Breaker), which should also be cited. On the other hand, the Clec ribozyme was only found in rodents and not primates and is thus not a human ribozyme and should be noted as such.

We thank this Reviewer for his/her input and acknowledgment of this work. To improve the manuscript, we have included the genomic locations in Figure 1A, Figure 6A and Figure 6C. And we have tested the activity of representative variants found in the human genome and discussed the activity of the variants in other primates. All suggested publications are now properly cited.

Line 62-66: It has been shown that single nucleotide polymorphism (SNP) in CPEB3 ribozyme was associated with an enhanced self-cleavage activity along with a poorer episodic memory (14). Inhibition of the highly conserved CPEB3 ribozyme could strengthen hippocampal-dependent long-term memory (15, 16). However, little is known about the other human self-cleaving ribozymes.

Line 474-501: Homology search of two TS-like ribozymes. To locate close homologs of the two TS-like ribozymes, we performed cmsearch based on a covariance model (38) built on the sequence and secondary structural profiles. In the human genome, we got 1154 and 4 homolog sequences for LINE-1-rbz and OR4K15-rbz, respectively. For OR4K15-rbz, there was an exact match located at the reverse strand of the exon of OR4K15 gene (Figure 6A). The other 3 homologs of OR4K15-rbz belongs to the same olfactory receptor family 4 subfamily K (Figure 6C). However, there was no exact match for LINE-1-rbz (Figure 6A). Interestingly, a total of 1154 LINE-1-rbz homologs were mapped to the LINE-1 retrotransposon according to the RepeatMasker (http://www.repeatmasker.org) annotation. Figure 6B showed the distribution of LINE-1-rbz homologs in different LINE-1 subfamilies in the human genome. Only three subfamilies L1PA7, L1PA8 and L1P3 (L1PA7-9) can be considered as abundant with LINE-1-rbz homologs (>100 homologs per family). The consensus sequences of all homologs obtained are shown in Figure 6D. In order to investigate the self-cleavage activity of these homologs, we mainly focused on the mismatches in the more conserved internal loops. The major differences between the 5 consensus sequences are the mismatches in the first internal loop. The widespread A12C substitution can be found in majority of LINE-1-rbz homologs, this substitution leads to a one-base pair extension of the second stem (P2) but almost no activity (RA’: 0.03) based on our deep mutational scanning result. Then we selected 3 homologs without A12C substitution for LINE-1-rbz for in vitro cleavage assay (Figure 6E). But we didn’t observe significant cleavage activity, this might be caused by GU substitutions in the stem region. For 3 homologs of OR4K15-rbz, we only found one homolog of OR4K15 with pronounced self-cleavage activity (Figure 6F). In addition, we performed similar bioinformatic search of the TS-like ribozymes in other primate genomes. Similarly, the majority (15 out of 18) of primate genomes have a large number of LINE-1 homologs (>500) and the remaining three have essentially none. However, there was no exact match. Only one homolog has a single mutation (U38C) in the genome assembly of Gibbon (Figure S15). The majority of these homologs have 3 or more mismatches (Figure S15). For OR4K15-rbz, all representative primate genomes contain at least one exact match of the OR4K15-rbz sequence.

Line 598-602: According to the bioinformatic analysis result, there are some TS-like ribozymes (one LINE-1-rbz homolog in the Gibbon genome, and some OR4K15-rbz homologs) with in vitro cleavage activity in primate genomes. Unlike the more conserved CPEB3 ribozyme which has a clear function, the function of the TS-like ribozymes is not clear, as they are not conserved, belong to the pseudogene or located at the reverse strand.

(2) The authors present the story as a discovery of a new RNA catalytic motif. This is unfounded. As the authors point out, the catalytic domain is very similar to the Twister Sister (or "TS") ribozyme. In fact, there is no appreciable difference between these and TS ribozymes, except for the missing peripheral domains. For example, the env33 sequence in the Weinberg et al. 2015 NCB paper shows the same sequences in the catalytic core as the LINE1 ribozyme, making the LINE1 ribozyme a TS-like ribozyme in every way, except for the missing peripheral domains. Thus these are not new ribozymes and should not have a new name. A more appropriate name should be TS-like or TS-min ribozymes. Renaming the ribozymes to lanterns is misleading.

Although we observed some differences in mutational effects, we agree with the reviewer that it is more appropriate to call them TS-like ribozymes. We have replaced all “lantern ribozyme” with “TS-like ribozyme” as suggested.

(3) In light of 2) the story should be refocused on the fact the authors discovered that the OR4K15 and LINE1 are both TS-like ribozymes. That is very exciting and is the real contribution of this work to the field.

We thank this Reviewer for their acknowledgement of this work. To improve the manuscript, we have re-named the ribozymes as suggested.

(4) Given the slow self-scission of the OR4K15 and LINE1 ribozymes, the discussion of the minimal domains should be focused on the role of peripheral domains in full-length TS ribozymes. Peripheral domains have been shown to greatly speed up hammerhead, HDV, and hairpin ribozymes. This is an opportunity to show that the TS ribozymes can do the same and the authors should discuss the contribution of peripheral domains to the ribozyme structure and activity. There is extensive literature on the contribution of a tertiary contact on the speed of self-scission in hammerhead ribozymes, in hairpin ribozyme it's centered on the 4-way junction vs 2-way junction structure, and in HDVs the contribution is through the stability of the J1/2 region, where the stability of the peripheral domain can be directly translated to the catalytic enhancement of the ribozymes.

We appreciate your question and the valuable suggestions provided. We have included the citations and discussion about the peripheral domains in other ribozymes.

Line 570-576: Thus, a more sophisticated structure along with long-range interactions involving the SL4 region in the twister sister ribozyme must have helped to stabilize the catalytic region for the improved catalytic activity. Similarly, previous studies have demonstrated that peripheral regions of hammerhead (49), hairpin (50) and HDV (51, 52) ribozymes could greatly increase their self-cleavage activity. Given the importance of the peripheral regions, absence of this tertiary interaction in the TS-like ribozyme may not be able to fully stabilize the structural form generated from homology modelling.

(5) The argument that these are the smallest self-cleaving ribozymes is debatable. LÃ1/4nse et al (NAR 2017) found some very small hammerhead ribozymes that are smaller than those presented here, but the authors suggest only working as dimers. The human ribozymes described here should be analyzed for dimerization as well (e.g., by native gel analysis) particularly because the authors suggest that there are no peripheral domains that stabilize the fold. Furthermore, Riccitelli et al. (Biochemistry) minimized the HDV-like ribozymes and found some in metagenomic sequences that are about the same size as the ones presented here. Both of these papers should be cited and discussed.

We apologize for any confusion caused by our previous statement. To clarify, we highlighted “35 and 31 nucleotides only” because 46 and 47 nt contain the variable hairpin loops which are not important for the catalytic activity. By comparing the conserved segments, the TS-like ribozyme discussed in this paper is the shortest with the simplest secondary structure. And we have replaced the terms “smallest” and “shortest” with “simplest” in our manuscript. The title has been changed to “Minimal twister sister (TS)-like self-cleaving ribozymes in the human genome revealed by deep mutational scanning”. All the publications mentioned have been cited and discussed. Regarding possible dimerization, we did not find any evidence but would defer it to future detailed structural analysis to be sure.  

Line 605-608: Previous studies also have revealed some minimized forms of self-cleaving ribozymes, including hammerhead (19, 53) and HDV-like (54) ribozymes. However, when comparing the conserved segments, they (>= 36 nt) are not as short as the TS-like ribozymes (31 nt) found here.

(6) The authors present homology modeling of the OR4K15 and LINE1 ribozymes based on the crystal structures of the TS ribozymes. This is another point that supports the fact that these are not new ribozyme motifs. Furthermore, the homology model should be carefully discussed as a model and not a structure. In many places in the text and the supplement, the models are presented as real structures. The wording should be changed to carefully state that these are models based on sequence similarity to TS ribozymes. Fig 3 would benefit from showing the corresponding structures of the TS ribozymes.

We thank the reviewer for pointing these out and we have already fixed them. We have replaced all “lantern ribozyme” with “TS-like ribozyme” as suggested. The term “Modelled structures” were used for representing the homology model. And we have included the TS ribozyme structure in Fig 3.

Reviewer #2 (Public Review):

Summary:

This manuscript applies a mutational scanning analysis to identify the secondary structure of two previously suggested self-cleaving ribozyme candidates in the human genome. Through this analysis, minimal structured and conserved regions with imminent importance for the ribozyme's activity are suggested and further biochemical evidence for cleavage activity are presented. Additionally, the study reveals a close resemblance of these human ribozyme candidates to the known self-cleaving ribozyme class of twister sister RNAs. Despite the high conservation of the catalytic core between these RNAs, it is suggested that the human ribozyme examples constitute a new ribozyme class. Evidence for this however is not conclusive.

Strengths:

The deep mutational scanning performed in this study allowed the elucidation of important regions within the proposed LINE-1 and OR4K15 ribozyme sequences. Part of the ribozyme sequences could be assigned a secondary structure supported by covariation and highly conserved nucleotides were uncovered. This enabled the identification of LINE-1 and OR4K15 core regions that are in essence identical to previously described twister sister self-cleaving RNAs.

Weaknesses:

I am skeptical of the claim that the described catalytic RNAs are indeed a new ribozyme class. The studied LINE-1 and OR4K15 ribozymes share striking features with the known twister sister ribozyme class (e.g. Figure 3A) and where there are differences they could be explained by having tested only a partial sequence of the full RNA motif. It appears plausible, that not the entire "functional region" was captured and experimentally assessed by the authors.

We thank this Reviewer for his/her input and acknowledgment of this work. Because a similar question was raised by reviewer 1, we decided to name the ribozymes as TS-like ribozymes. Regarding the entire regions, we conducted mutational scanning experiments at the beginning of this study. The relative activity distributions (Figure 1B, 1C) have shown that only parts of the sequence contributes to the self-cleavage activity. That is the reason why we decided to focus on the parts of the sequence afterwards.

They identify three twister sister ribozymes by pattern-based similarity searches using RNA-Bob. Also comparing the consensus sequence of the relevant region in twister sister and the two ribozymes in this paper underlines the striking similarity between these RNAs. Given that the authors only assessed partial sequences of LINE-1 and OR4K15, I find it highly plausible that further accessory sequences have been missed that would clearly reveal that "lantern ribozymes" actually belong to the twister sister ribozyme class. This is also the reason I do not find the modeled structural data and biochemical data results convincing, as the differences observed could always be due to some accessory sequences and parts of the ribozyme structure that are missing.

We appreciate the reviewer for raising this question. As we explained in the last question, we now called the ribozymes as TS-like ribozymes. We also emphasize that the relative activity data of the original sequences have indicated that the other part did not make any contribution to the activity of the ribozyme. The original sequences provided in the Science paper (Salehi-Ashtiani et al. Science 2006) were generated from biochemical selection of the genomic library. It did not investigate the contribution of each position to the self-cleavage activity.

Highly conserved nucleotides in the catalytic core, the need for direct contacts to divalent metal ions for catalysis, the preference of Mn2+ oder Mg2+ for cleavage, the plateau in observed rate constants at ~100mM Mg2+, are all characteristics that are identical between the proposed lantern ribozymes and the known twister sister class.

The difference in cleavage speed between twister sister (~5 min-1) and proposed lantern ribozymes could be due to experimental set-up (true single-turnover kinetics?) or could be explained by testing LINE-1 or OR4K15 ribozymes without needed accessory sequences. In the case of the minimal hammerhead ribozyme, it has been previously observed that missing important tertiary contacts can lead to drastically reduced cleavage speeds.

We thank the reviewer for this question. We now called the ribozymes as TS-like ribozymes. As we explained in the last question, the relative activity data of the original sequences have proven that the other part did not make any contribution to the activity of the ribozyme. Moreover, we have tested different enzyme to substrate ratios to achieve single turn-over kinetics (Figure S13). The difference in cleavage speed should be related to the absence of peripheral regions which do not exist in the original sequences of the LINE-1 and OR4K15 ribozyme. We have included the publications and discussion about the peripheral domains in other ribozymes.

Line 458-463: The kobs of LINE-1-core was ~0.05 min-1 when measured in 10mM MgCl2 and 100mM KCl at pH 7.5 (Figure S13). Furthermore, the single-stranded ribozymes exhibited lower kobs (~0.03 min-1 for LINE-1-rbz) (Figure S14) when comparing with the bimolecular constructs. This confirms that the stem loop region SL2 does not contribute much to the cleavage activity of the TS-like ribozymes.

Line 570-576: Thus, a more sophisticated structure along with long-range interactions involving the SL4 region in the twister sister ribozyme must have helped to stabilize the catalytic region for the improved catalytic activity. Similarly, previous studies have demonstrated that peripheral regions of hammerhead (49), hairpin (50) and HDV (51, 52) ribozymes could greatly increase their self-cleavage activity. Given the importance of the peripheral regions, absence of this tertiary interaction in the TS-like ribozyme may not be able to fully stabilize the structural form generated from homology modelling.

Reviewer 2: ( Recommendations For The Authors):

Major points

It would have made it easier to connect the comments to text passages if the submitted manuscript had page numbers or even line numbers.

We thank the reviewer for pointing this out and we have already fixed it.

In the introduction: "...using the same technique, we located the functional and base-pairing regions of..." The use of the adjective functional is imprecise. Base-paired regions are also important for the function, so what type of region is meant here? Conserved nucleotides?

We thank the reviewer for pointing this out. We were describing the regions which were essential for the ribozyme activity. And we have defined the use of “functional region” in introduction.

Line 95: we located the regions essential for the catalytic activities (the functional regions) of LINE-1 and OR4K15 ribozymes in their original sequences.

In their discussion, the authors mention the possible flaws in their 3D-modelling in the absence of Mg2+. Is it possible to include this divalent metal ion in the calculations?

We thank the reviewer for this question. Currently, BriQ (Xiong et al. Nature Communications 2021) we used for modeling doesn’t include divalent metal ion in modeling.

Xiong, Peng, Ruibo Wu, Jian Zhan, and Yaoqi Zhou. 2021. “Pairing a High-Resolution Statistical Potential with a Nucleobase-Centric Sampling Algorithm for Improving RNA Model Refinement.” Nature Communications 12: 2777. doi:10.1038/s41467-021-23100-4.

Abstract:

It is claimed that ribozyme regions of 46 and 47 nt described in the manuscript resemble the shortest known self-cleaving ribozymes. This is not correct. In 1988, hammerhead ribozymes in newts were first discovered that are only 40 nt long.

We apologize for any confusion caused by our previous statement. To clarify, we highlighted “35 and 31 nucleotides only” as 46 and 47 nt contain the variable hairpin loops which are not important for the catalytic activity. By comparing the conserved segments, the TS-like ribozyme discussed in this paper is the shortest with the simplest secondary structure. And we have replaced the terms “smallest” and “shortest” with “simplest” in our manuscript. The title has been changed to “Minimal TS-like self-cleaving ribozyme revealed by deep mutational scanning”.

The term "functional region" is, to my knowledge, not a set term when discussing ribozymes. Does it refer to the catalytic core, the cleavage site, the acid and base involved in cleavage, or all, or something else? Therefore, the term should be 1) defined upon its first use in the manuscript and 2) probably not be used in the abstract to avoid confusion to the reader.

We apologize for any confusion caused by our previous statement. To clarify, we have changed the term “functional region” in abstract. And we have defined the use of “functional region” in introduction.

Line 34-37: We found that the regions essential for ribozyme activities are made of two short segments, with a total of 35 and 31 nucleotides only. The discovery makes them the simplest known self-cleaving ribozymes. Moreover, the essential regions are circular permutated with two nearly identical catalytic internal loops, supported by two stems of different lengths.

Line 95: we located the regions essential for the catalytic activities (the functional regions) of LINE-1 and OR4K15 ribozymes in their original sequences.

The choice of the term "non-functional loop" in the abstract is a bit unfortunate. The loop might not be important for promoting ribozyme catalysis by directly providing, e.g. the acid or base, but it has important structural functions in the natural RNA as part of a hairpin structure.

We thank the reviewer for pointing this out and we have re-phrased the sentences.

Line 33-34: We found that the regions essential for ribozyme activities are made of two short segments, with a total of 35 and 31 nucleotides only.

Line 283: Removing the peripheral loop regions (Figures 1B and 1C) allows us to recognize that the secondary structure of OR4K15-rbz is a circular permutated version of LINE-1-rbz.

Results:

Please briefly explain CODA and MC analysis when first mentioned in the results (Figure (1) The more detailed explanation of these terms for Figure 2 could be moved to this part of the results section (including explanations in the figure legend).

We thank the reviewer for pointing this out and we included a brief explanation.

Line 150-154: CODA employed Support Vector Regression (SVR) to establish an independent-mutation model and a naive Bayes classifier to separate bases paired from unpaired (26). Moreover, incorporating Monte-Carlo simulated annealing with an energy model and a CODA scoring term (CODA+MC) could further improve the coverage of the regions under-sampled by deep mutations.

Please indicate the source of the human genomic DNA. Is it a patient sample, what type of tissue, or is it an immortalized cell line? It is not stated in the methods I believe.

We thank the reviewer for pointing this out. According to the original Science paper (Salehi-Ashtiani et al. Science 2006), the human genomic DNA (isolated from whole blood) was purchased from Clontech (Cat. 6550-1). In our study, we directly employed the sequences provided in Figure S2 of the Science paper for gene synthesis. Thus, we think it is unnecessary to mention the source of genomic DNA in the methods section of our paper.  

Please also refer to the methods section when the calculation of RA and RA' values is explained in the main text to avoid confusion.

We thank the reviewer for pointing this out and we have fixed it.

Line 207-208: Figure 2A shows the distribution of relative activity (RA’, measured in the second round of mutational scanning) (See Methods) of all single mutations

For OR4K15 it is stated that the deep mutational scanning only revealed two short regions as important. However, there is another region between approx. 124-131 nt and possibly even at positions 47 and 52 (to ~55), that could contribute to effective RNA cleavage, especially given the library design flaws (see below) and the lower mutational coverage for OR4K15. A possible correlation of the mutations in these regions is even visible in the CODA+MC analysis shown in Figure 1D on the left. Why are these regions ignored in ongoing experiments?

We thank the reviewer for this question. As shown in Table S1, although the double mutation coverage of OR4K15-ori was low (16.2 %), we got 97.6 % coverage of single mutations. The relative activity of these single mutations was enough to identify the conserved regions in this ribozyme. Mutations at the positions mentioned by the reviewer did not lead to large reductions in relative activity. Since the relative activity of the original sequence is 1, we presumed that only positions with average relative activity much lower than 1 might contribute to effective cleavage.

Regarding the corresponding correlation of mutations in CODA+MC, they are considered as false positives generated from Monte Carlo simulated annealing (MC), because lack of support from the relative activity results.

Have the authors performed experiments with their "functional regions" in comparison to the full-length RNA or partial truncations of the full-length RNA that included, in the case of OR4K15, nt 47-131? Also for LINE-1 another stem region was mentioned (positions 14-18 with 30-34) and two additional base pairs. Were they included in experiments not shown as part of this manuscript?

We appreciate the reviewer for raising this question. We only compared the full-length or partial truncations of the LINE-1 ribozyme. Since the secondary structure predicted from OR4K15-ori data was almost the same as LINE-1, we didn’t perform deep mutagenesis on the partial truncation of the OR4K15. However, the secondary structure of OR4K15 was confirmed by further biochemical experiments.   

Regarding the second question, the additional base pairs were generated by Monte Carlo simulated annealing (MC). They are considered as false positives because of low probabilities and lack of support from the deep mutational scanning results. The appearance of false positives is likely due to the imperfection of the experiment-based energy function employed in current MC simulated annealing. 

Are there other examples in the literature, where error-prone PCR generates biases towards A/T nucleotides as observed here? Please cite!

We thank the reviewer for pointing this out and we have included the corresponding citation.

Line 161-162: The low mutation coverage for OR4K15-ori was due to the mutational bias (27, 28) of error-prone PCR (Supplementary Figures S1, S2, S3 and S4).

Line 170-171: whose covariations are difficult to capture by error-prone PCR because of mutational biases (27, 28).

The authors mention that their CODA analysis was based on the relative activities of 45,925 and 72,875 mutation variants. I cannot find these numbers in the supplementary tables. They are far fewer than the read numbers mentioned in Supplementary Table 2. How do these numbers (45,925 and 72,875) arise? Could the authors please briefly explain their selection process?

We apologize for any confusion caused by our previous statement. Our CODA analysis only utilized variants with no more than 3 mutations. The number listed in the supplementary tables is the total number of the variants. To clarify, we have included a brief explanation for these numbers.

Line 203-204: We performed the CODA analysis (26) based on the relative activities of 45,925 and 72,875 mutation variants (no more than 3 mutations) obtained for the original sequence and functional region of the LINE-1 ribozyme, respectively.

What are the reasons the authors assume their findings from LINE-1 can be used to directly infer the structure for OR4K15? (Third section in results, last paragraph)

We apologize for any confusion caused by our previous statement. We meant to say that the consistency between LINE-1-rbz and LINE-1-ori results suggested that our method for inferring ribozyme structure was reliable. Thus, we employed the same method to infer the structure of the functional region of OR4K15. To clarify, we have re-phrased the sentence.   

Line 259-261: The consistent result between LINE-1-rbz and LINE-1-ori suggested that reliable ribozyme structures could be inferred by deep mutational scanning. This allowed us to use OR4K15-ori to directly infer the final inferred secondary structure for the functional region of OR4K15.

There are several occasions where the authors use the differences between the proposed lantern ribozymes and twister sister data as reasons to declare LINE-1 and OR4K15 a new ribozyme class. As mentioned previously, I am not convinced these differences in structure and biochemical results could not simply result from testing incomplete LINE-1 and OR4K15 sequences.

We apologize for any confusion caused by our previous statement. Despite we observed some differences in mutational effects, we agree with the reviewer that it is not convincing to claim them as a new ribozyme class. We have replaced all “lantern ribozyme” with “TS-like ribozyme” as the reviewer 1 suggested.

The authors state, that "the result confirmed that the stem loop SL2 region in LINE-1 and OR4K15 did not participate in the catalytic activity". To draw such a conclusion a kinetic comparison between a construct that contains SL2 and does not contain SL2 would be necessary. The given data does not suffice to come to this conclusion.

We appreciate the reviewer for raising this question. To address this, we performed gel-based kinetic analysis of these two ribozymes (Figure S14).

Line 458-462: The kobs of LINE-1-core under single-turnover condition was ~0.05 min-1 when measured in 10mM MgCl2 and 100mM KCl at pH 7.5 (Figure S13). Only a slightly lower value of  kobs (~0.03 min-1) was observed for LINE-1-rbz (Figure S14). This confirms that the stem loop region SL2 does not contribute to the cleavage activity of the TS-like ribozymes.

Construct/Library design:

The last 31 bp in the OR4K15 ribozyme template sequence are duplicated (Supplementary Table 4). Therefore, there are 2 M13 fwd binding sites and several possible primer annealing sites present in this template. This could explain the lower yield for the mutational analysis experiments. Did the authors observe double bands in their PCR and subsequent analysis? The experiments should probably be repeated with a template that does not contain this duplication. Alternatively, the authors should explain, why this template design was chosen for OR4K15.

We apologize for this mistake during writing. Our construct design for OR4K15 contains only one M13F binding site. We thank the reviewer for pointing this out and we have fixed the error.

Figure 5B: Where are the bands for the OR4K15 dC-substrate? They are not visible on the gel, so one has to assume there was no substrate added, although the legend indicates otherwise.

Also this figure, please indicate here or in the methods section what kind of marker was used. In panels A and B, please label the marker lanes.

We apologize for this mistake and we have repeated the experiment. The marker lane was removed to avoid confusion caused by the inappropriate DNA marker. 

The authors investigated ribozyme cleavage speeds by measuring the observed rate constants under single-turnover conditions. To achieve single-turnover conditions enzyme has to be used in excess over substrate. Usually, the ratios reported in the literature range between 20:1 (from the authors citation list e.g.: for twister sister (Roth et al 2014) and hatchet (Li et al. 2015)) or even ~100:1 (for pistol: Harris et al 2015, or others https://www.sciencedirect.com/science/article/pii/S0014579305002061). Can the authors please share their experimental evidence that only 5:1 excess of enzyme over the substrate as used in their experiments truly creates single-turnover conditions?

We greatly appreciate the Reviewer for raising this question. To address this, we performed kinetic analysis using different enzyme to substrate ratios (Figure S13). There is not too much difference in kobs, except that kobs reach the highest value of 0.048 min-1 when using 100:1 excess of enzyme over the substrate. 

Line 458-460: The kobs of LINE-1-core under single-turnover condition was ~0.05 min-1 when measured in 10mM MgCl2 and 100mM KCl at pH 7.5 (Figure S13).

Citations:

In the introduction citation number 12 (Roth et al 2014) is mentioned with the CPEB3 ribozyme introduction. This is the wrong citation. Please also insert citations for OR4K15 and IGF1R and LINE-1 ribozyme in this sentence.

We thank the reviewer for pointing this out and we now have fixed it.

Also in the introduction, a hammerhead ribozyme in the 3' UTR of Clec2 genes is mentioned and reference 16 (Cervera et al 2014) is given, I think it should be reference 9 (Martick et al 2008)

We thank the reviewer for pointing this out and we now have fixed it.

In the results section it is stated that, "original sequences were generated from a randomly fragmented human genomic DNA selection based biochemical experiment" citing reference 12. This is the wrong reference, as I could not find that Roth et al 2014 describe the use of such a technique. The same sentence occurs in the introduction almost verbatim (see also minor points).

We thank the reviewer for pointing this out and we now have fixed it.

Minor points

Headline:

Either use caps for all nouns in the headline or write "self-cleaving ribozyme" uncapitalized

We thank the reviewer for pointing this out and we now have fixed it.

Abstract:

1st sentence: in "the" human genome

"Moreover, the above functional regions are..." - the word "above" could be deleted here

"named as lantern for their shape"- it should be "its shape"

"in term of sequence and secondary structure"- "in terms"

"the nucleotides at the cleavage sites" - use singular, each ribozyme of this class has only one cleavage site

We thank the reviewer for pointing these out and we now have fixed them.

Introduction:

Change to "to have dominated early life forms"

Change to "found in the human genome"

Please write species names in italics (D. melanogaster, B. mori)

Please delete "hosting" from "...are in noncoding regions of the hosting genome"

Please delete the sentence fragment/or turn it into a meaningful sentence: "Selection-based biochemical experiments (12).

Change to "in terms of sequence and secondary structure, suggesting a more"

Please reword the last sentence in the introduction to make clear what is referred to by "its", e.g. probably the homology model of lantern ribozyme generated from twister sister ribozymes?

Please refer to the appropriate methods section when explaining the calculation of RA and RA'.

We thank the reviewer for pointing these out and we now have fixed them.

The last sentence of the second paragraph in the second section of the results states that the authors confirmed functional regions for LINE-1 and OR4K15, however, until that point the section only presents data on LINE-1. Therefore, OR4K15 should not be mentioned at the end of this paragraph.

In response to the reviewer's suggestions, we have removed OR4K15 from this paragraph.

Line 225-228: The consistency between base pairs inferred from deep mutational scanning of the original sequences and that of the identified functional regions confirmed the correct identification of functional regions for LINE-1 ribozyme.

Change to "Both ribozymes have two stems (P1, P2), to internal loops ..."

We thank the reviewer for pointing this out and we now have fixed it.

The section naming the "functional regions" of LINE-1 and OR4K15 lantern ribozymes should be moved after the section in which the circular permutation is shown and explained. Therefore, the headline of section three should read "Consensus sequence of LINE-1 and OR4K15 ribozymes" or something along these lines.

We thank the reviewer for pointing this out and we now have fixed it.

Line 308-309: Given the identical lantern-shaped regions of the LINE-1-rbz and OR4K15-rbz ribozyme, we named them twister sister-like (TS-like) ribozymes.

The statement on the difference between C8 in OR4K15 and U38 in LINE-1 should be further classified. As U38 is only 95% conserved. Is it a C in those other instances or do all other nucleotide possibilities occur? Is the high conservation in OR4K15 an "artifact" of the low mutation rate for this RNA in the deep mutational scanning?

We thank the reviewer for this question. Yes, the high conservation in OR4K15 an "artifact" of the low mutation rate for this RNA in the deep mutational scanning. That is why RA’ value is more appropriate to describe the conservation level of each position. We also mentioned this in the manuscript:

Line 287-288: The only mismatch U38C in L1 has the RA’ of 0.6, suggesting that the mismatch is not disruptive to the functional structure of the ribozyme.

Section five, first paragraph: instead of "two-stranded LINE-1 core" use the term "bimolecular", as it is more commonly used.

We thank the reviewer for pointing this out and we now have changed it.

Figure caption 3 headline states "Homology modelled 3D structure..."but it also shows the secondary structures of LINE1, OR4K15 and twister sister examples.

We thank the reviewer for pointing this out and we now have removed “3D”.

In Figure 3C, we see a nucleobase labeled G37, however in the secondary structure and sequence and 3D structural model there is a C37 at this position. Please correct the labeling.

We thank the reviewer for pointing this out and we now have fixed it.

Section 7 "To address the above question..." please just repeat the question you want to address to avoid any confusion to the reader.

We thank the reviewer for pointing these out and we have re-phrased this sentence.

Line 364: Considering the high similarity of the internal loops, we further investigated the mutational effects on the internal loop L1s.

Please rephrase the sentence "By comparison, mutations of C62 (...) at the cleavage site did not make a major change on the cleavage activity...", e.g. "did not lead to a major change" etc.

Section 8, first paragraph: This result further confirms that the RNA cleavage in lantern...", please delete "further"

Change to "analogous RNAs that lacked the 2' oxygen atom in the -1 nucleotide"

Methods

Change to "We counted the number of reads of the cleaved and uncleaved..."

Change to "...to produce enough DNA template for in vitro transcription."

Change to "The DNA template used for transcription was used..." (delete while)

We thank the reviewer for pointing these out and we now have fixed them.

Supplement

All supplementary figures could use more detailed Figure legends. They should be self-explanatory.

Fig S1/S2: how is "mutation rate" defined/calculated?

We thank the reviewer for pointing this out and we now have added a short explanation. The mutation rate was calculated as the proportion of mutations observed at each position for the DNA-seq library.

Fig S3/S4: axis label "fraction", fraction of what? How calculated?

We thank the reviewer for pointing this out and we now have added a short explanation. The Y axis “fraction” represents the ratio of each mutation type observed in all variants.

Fig S5: RA and RA' are mentioned in the main text and methods, but should be briefly explained again here, or it should be clearly referred to the methods. Also, the axis label could be read as average RA' divided by average RA. I assume that is not the case. I assume I am looking at RA' values for LINE-1 rbz and RA values for LINE-1-ori? Also, mention that only part of the full LINE-1-ori sequence is shown...

We thank the reviewer for pointing this out and we have now added a short explanation. The Y axis represents RA’ for LINE-1-rbz, or RA for LINE-1-ori. The part shown is the overlap region between LINE-1-rbz and LINE-1-ori. We apologize for any confusion caused by our previous statement.

Fig S9 the magenta for coloring of the scissile phosphate is hard to see and immediately make out.

We thank the reviewer for pointing this out and we now have added a label to the scissile phosphate.

Fig S10: Why do the authors only show one product band here? Instead of both cleavage fragments as in Figure 5?

We thank the reviewer for this question. We purposely used two fluorophores (5’ 6-FAM, 3’ TAMRA) to show the two product bands in Figure 5. In Fig S10, long-time incubation was used to distinguish catalysis based self-cleavage from RNA degradation. This figure was prepared before the purchasing of the substrate used in Figure 5. The substrate strand used in Fig S10 only have one fluorophore (5’ 6-FAM) modification. And the other product was too short to be visualized by SYBR Gold staining.

Fig S13: please indicate meaning of colors in the legend (what is pink, blue, grey etc.)

Please change to "RtcB ligase was used to capture the 3' fragment after cleavage...."

We thank the reviewer for pointing this out and we now have fixed it.

The writer informs us of how the sitcom is structured. She proves here what makes the sitcom different from a traditional marvel. This information further supports her main idea.

Reviewer #1 (Public review):

Summary:

Arman Angaji and his team delved into the intricate world of tumor growth and evolution, utilizing a blend of computer simulations and real patient data from liver cancer.

Strengths:

Their analysis of how mutations and clones are distributed within tumors revealed an interesting finding: tumors don't just spread from their edges as previously believed. Instead, they expand both from within and the edges simultaneously, suggesting a unique growth mode. This mode naturally indicates that external forces may play a role in cancer cells dispersion within the tumor. Moreover, their research hints at an intriguing phenomenon - the high death rate of progenitor cells and extremely slow pace in growth in the initial phase of tumor expansion. Understanding this dynamic could significantly impact our comprehension of cancer development.

Weaknesses:

It's important to note, however, that this study relies on specific computer models, metrics derived from inferred clones, and a limited number of patient data. While the insights gained are promising, further investigation is essential to validate these findings. Nonetheless, this work opens up exciting avenues for comprehending the evolution of cancers.

Comments on revised submission:

The authors have effectively addressed my concerns. This revision is excellent.

Author response:

The following is the authors’ response to the original reviews.

Public reports:

In the public reports there is only one point we would like to discuss. It concerns our use of a computational model to analyse spatial tumour growth. Citing from the eLife assessment, which reflects several comments of the referees:

The paper uses published data and a proposed cell-based model to understand how growth and death mechanisms lead to the observed data. This work provides an important insight into the early stages of tumour development. From the work provided here, the results are solid, showing a thorough analysis. However, the work has not fully specified the model, which can lead to some questions around the model’s suitability.

The observables we use to determine the (i) growth mode and the (ii) dispersion of cells are modelindependent. The method to determine the (iii) rate of cell death does not use a spatial model. Throughout, our computational model of spatial growth is not used to analyze data. Instead, it is used to check that the observables we use can actually discriminate between different growth modes given the limitations of the data. We have expanded the description of the computational model in the revised version, and have released our code on Github. However, the conclusions we reach do not rely on a computational model. Instead, where we estimate parameters, we use population dynamics as described in section S5. The other observables are parameter free and model-independent. We view this as a strength of our approach.

Recommendations for the authors:

Reviewer #1:

(1.1) In Figure 1, the data presented by Ling et al. demonstrate a distinctive “comb” pattern. While this pattern diverges from the conventional observations associated with simulated surface growth, it also differs from the simulated volume growth pattern. Is this discrepancy attributable to insufficient data? Alternatively, could the emergence of such a comb-like structure be feasible in scenarios featuring multiple growth centers, wherein clones congregate into spatial clusters?

We are unsure what you are referring to. One possibility is you refer to the honey-comb structure formed by the samples of the Ling et al. data shown in Fig. 1A of the main text. This is an artefact arising from the cutting of the histological cut into four quadrants, see Fig. S1 in the SI of Ling et al. The perceived horizontal and vertical “white lines” in our Fig. 1A stems from the lack of samples near the edges of these quadrants. We have added this information to the figure caption.

An alternative is you are referring to the peaks in Fig 2A of the main text. The three of these peaks indeed stem from individual clones. We have placed additional figures in the SI (S2 B and S2 C) to disentangle the contribution from different clones. The peaks have a simple explanation: each clone contributes the same weight to the histogram. If a clone only has few offspring, this statistical weight is concentrated on a few angles only, see SI Figure S2 B.

(1.2) I am not sure why there are two sections about “Methods” in the main text: Line 50 as well as Line 293. Furthermore, the methods outlined in the main paper lack the essential details necessary for readers to navigate through the critical aspects of their analysis. While these details are provided in the Supplementary Information, they are not adequately referenced within the methods section of the main text. I would recommend that the authors revise the method sections of the main text to include pertinent descriptions of key concepts or innovations, while also directing readers to the corresponding supplementary method section for further elucidation.

We have merged the Section “Materials and Methods” at the end of the main text with the SI description of the data in SI 4.2 and placed a reference to this material in the main body.

(1.3) The impact of the particular push method (proposed in the model) on the resultant spatial arrangement of clones remains unclear. For instance, it’s conceivable that employing a different pushing method (for example, with more strict constraints on direction) could yield a varied pattern of spatial diversity. Furthermore, there is ambiguity regarding the criteria for determining the sequence of the queue housing overlapping cells.

Regarding the off-lattice dynamics we use, there are indeed many variants one could use. In nonexhaustive trials, we found that the details of the off-lattice dynamics did not affect the results. The reason may be that at each computational step, each cell only moves a very small amount, and differences in the dynamics tend to average out over time.

We deliberately do not give constraints on the direction. Such constraints emerge in lattice-based models (when preferred directions arise from the lattice symmetry), but these are artifacts of the lattice.

At cell division the offspring is placed in a random direction next to the parent regardless of whether this introduces an overlap. Cells then push each other along the axis connecting their two centers of mass – unlike in lattice based models a sequence of pushes does not propagate through the tumor straight away but sets off of a cascade of pushes. Equal pushing of two cells (i.e. two initial displacements as opposed to pushing one of the two) results in the same patterns of directed, low dispersion surface and undirected, high dispersion volume growth but is much harder computationally as it reintroduces overlaps that have been resolved in the previous step.

We have rewritten the description of the pushing queue in the SI Section 1. The choice of the pushing sequence is somewhat arbitrary but we found that it also has no noticable effect on the growth mode. Maybe putting it in contrast to depth-first approaches helps to illustrate this: We tried two queueing schemes for iterating through overlapping cells, width-first and depth-first. In both cases, we begin by scanning a given cell’s (the root’s) neighborhood for overlaps and shuffle the list of overlapping neighbours. In a width-first approach we then add this list to the queue. Subsequent iterations append their lists of overlapping cells to the queue, such that we always resolve overlaps within the neighborhood of the root first. A depth-first approach follows a sequence of pushes by immediately checking a pushed cell’s neighborhood for new overlaps and adding these to the front of the queue (which works more like a stack then). This can be efficiently implemented by recursion but has no noticeable performance advantage and results in the same patterns of directed, low dispersion surface and undirected, high dispersion volume growth. In our opinion the width-first approach of first resolving overlaps in the immediate neighborhood is more intuitive, which is why we adopted it for our simulation model.

(1.4) For the example presented in S5.1, how can the author identify from genomic data that mutation 3 does not replace its ancestral clade mutation 2? In other words, if mutation 2, 3 and 4 are linked meaning clone 4 survives but 2 and 3 dies, how does one know if clone 3 dies before clone 2? I understand that this is a conceptual example, but if one cannot identify this situation from the real data, how can the clade turnover be computed?

Thank you for this comment, which points to an error of ours in the turnover example of the SI: Clade 3 does in fact replace 2 and contributes to the turnover! (The algorithm correctly annotated clade 3 as orphaned and computes a turnover of 3/15 for this example). We have corrected this.

In this example, it does not matter for the clade turnover whether clone 3 dies before clone 2. As long as its ancestor (clone 2) becomes extinct it adds to the clade turnover. The term “replaces” applies to the clade of 3 which has a surviving subclone and thereby eventually replaces clade 2. The clade turnover its solely based on the presence of the mutations (which define their clade) and not on the individual clones.

(1.5) After reviewing reference 24 (Li et al.), I noticed that the assertions made therein contradict the findings presented in S3 (Mutation Density on Rings). Specifically, Li et al. state that “peripheral regions not only accumulated more mutations, but also contained more changes in genes related to cell proliferation and cell cycle function” (Page 6) and “Phylogenetic trees show that branch lengths vary greatly with the long-branched subclones tending to occur in peripheral regions” (Page 4). However, upon re-analysis of their data, the authors demonstrated a decrease in mutation density near the surface. It is crucial to comprehend the underlying cause of such a disparity.

The reason for this disparity is the way Li et al. labelled samples as belonging to peripheral or central regions of the tumour. We have added a new figure in the SI to show this: Fig. S14 shows the number of mutations found in samples of Li et al. against their distances from the centre, along with the classification of samples as center/periphery given in Li et al. In the case of tumor T1, the classification of a sample in reference Li et al. does not agree with the distance from the center: samples classified as core are often more distant from the center than those classified as peripheral. Furthermore, Lewinsohn et al. (see below) show in their Fig. 5 that samples classified as ‘center’ by Li all fall into a single clade, and we believe this affects all results derived from this classification. For this reason, we do not consider the classification in reference 24 (Li et al.) further. We now briefly discuss this in Section S3.3.

(1.6) The authors consider coinciding mutations to occur when offspring clades align with an ancestral clade. Nevertheless, since multiple mutations can arise simultaneously in a single generation (such as kataegis), it becomes essential to discern its impact on clade turnover and, consequently, the estimation of d/b.

The mutational signatures found here show no sign of kataegis. Also, the number of polymorphic sites in the whole-exome data is small and the mutations are uniformly spread across the exome. The point is well taken, however, the method requires single mutations per generation. In practice, this can be achieved by subsampling a random part of the genome or exome (see [45]). We tested this point by processing the data from only a fraction of the exome; this did not change the results. In particular, Figure S30 shows the turnover-based inference for different subsampling rates L of the Ling et al. data. Subsampling of sites reduces the exome-wide mutation rate, the inferred rate scales linearly with L, as expected.

(1.7) I could not understand Step 2 in Section S2.1, an illustration may be helpful.

We have added figure S2 explaining the directional angle algorithm to Section S2.1 in the supplementary information.

(1.8) Figure S2, does a large rhoc lead to volume growth rather than surface growth, not the other way around?

Thank you for catching this mix-up!

Reviewer #2

I do have a few minor comments/questions, but I am confident the authors will be able to address them appropriately.

(2.1) Line 56: I am not sure what the units of “average read depth 74X” is in terms of SI units?

This number gives the number of sequence reads covering a particular nucleotide and is dimensionless. We have added this information.

(2.2) Lines 63 - 68: I am unsure what is meant by the terms “T1 of ca.” and “T2 of ca.”. Can these also be explained/defined please?

These refer to the approximate (circa) diameters of tumor 1 and tumor 2 in the data by Li et al. We have expanded the abbreviations.

(2.3) Line 69: I would like to see a more extensive description of the cell-based model here in the main text, such as how do the cells move. Moreover, do cells have a finite reach in space, do they have a volume/area?

We have expanded the model description in the main body of the paper and placed information there that previously was only in the SI.

(2.4) Line 76: You have said cells can “push” one another in your model. Do they also “pull” one another? Cell adhesion is know to contribute to tumour integrity - so this seems important for a model of this nature.

We have not implemented adhesive forces between pairs of cells so far. This would cause a higher pressure under cell growth (which can have important physiological consequences). However, the hard potential enforcing a distance between adjacent cells would still lead to cells pushing each other apart under population growth, so we expect to see the dispersion effect we discuss even when there is adhesion.

(2.5) Line 80-81: “due to lack of nutrient”. Is nutrient included in this work? It is my understanding it is not. No problem if so, it is just that this line makes it seem like it is and important. If it is not, the authors should mention this in the same sentence.

Thank you for pointing out this source of misunderstanding, your understanding is correct and we have modified the text to remove the ambiguity.

(2.6) Line 94-95: Since you are interested in tissue growth, recent work has indicated how the cell boundary (and therefore tissue boundary) description influences growth. Please also be sure to indicate this when you describe the model.

We presume you refer to the recent paper by Lewinsohn et al. (Nature Ecology and Evolution, 2023), which reports a phylogenetic analysis based on the Li et al. data. Lewinsohn et al. find that cells near the tumour boundary grow significantly faster than those in the tumour’s core. This is at variance with what we find; we were not aware of this paper at the time of submission. We now refer to this paper in the main text, and also have included a new section S3.4 in the SI accounting for this discrepancy. If you refer to a different paper, please let us know.

Briefly, we repeat the analysis of Lewinsohn et al., using their algorithm on artificial data generated by our model under volume growth. Samples were placed precisely like they were placed in the tumor analyzed by Li et al. We find that, even though the data was generated by volume growth, the algorithm of Lewinsohn et al. finds a signal of surface growth, in many cases even stronger compared to the signal which Lewinsohn et al. find in the empirical data. We have added subsection S3.4 with new figure S15 in the Supplementary Information.

(2.7) Line 107: “thus no evidence for enhanced cell growth near the edge of the tumour”. It is unclear to me how this tells us information relative to the tumour edge. It seems to me this is an artifact that at the edge of the tumour, there are less cells to compare with? Could you please expand on this a bit?

The direction angles tell us if new mutations arise predominantly radially outwards. With this observable, surface growth would lead to a non-uniform distribution of these angles even if we restrict the analysis to samples from the interior of the tumor (which, under surface growth, was once near the surface). So the effect is not linked to fewer cells for comparison. Also, we have checked the direction angles in simulations under different growth modes with the samples placed in the same way as in the data (see Figs. S3 and S4 right panels). We have expanded the text in the main text, section Results accordingly.

(2.8) I really enjoyed the clear explanation between lines 119 and 122 regarding cell dispersion!

Thank you!

(2.9) Figure 2B: Since you are looking at a periodic feature in theta, I would have expected the distribution to be periodic too, and therefore equal at theta=-180=180. Can you explain why it is different, please? Interestingly, you simulated data does seem to obey this!

The distribution of theta is periodic but the binning and midpoints of bins were chosen badly. We have replotted the diagram with bin boundaries that handle the edge-points -180/180 correctly. Thank you for pointing this out.

(2.10) Figure 3B: This plot does not have a title. Also, what do the red vertical lines in plots 3B, 3C and 3D indicate?

We have added the title. The red lines indicate the expectation values of the distributions.

(2.11) Figure 4: I am unsure how to read the plot in 4B. Also, what does the y-axis represent in 4C and 4D?

We have added explanations for 4B and have placed the labels for 4C and 4D in the correct position on the y-axes.

(2.12) Lines 194-199: you discuss your inferred parameters here, but you do not indicate how you inferred these parameters. May you please briefly mention how you inferred these, please?

These were inferred using the turnover method explained in the paragraph above, we have expanded the information. A full account is given in the SI Section S5.

(2.13) Line 258-260: “... mutagen (aristolochic acid) found in herbal traditional Chinese medicine and thought to cause liver cancer.” I do not see what this sentence adds to the work. Could you please be clearer with the claim you are making here?

Mutational signatures allow to infer underlying mutational processes. The strongest signature found in the data is associated with a mutagen that has in the past been used in traditional Chinese medicines. The patients from whom the tumours were biopsied were from China, so past exposure to this potent mutagen is possible. We are not making a big claim here, the mutational signature of aristolochic acid and its cancerogenic nature has been well studied and is referenced here. The result is interesting in our context because in one of the datasets (Li et al.) the signature is present in early (clonal) mutations but absent in later ones, allowing to make inferences from present data on the past. We have added the information that the patients were from China.

(2.14) In your Supplementary Information, S1, I believe your summation should not be over i, as you state in the following it is over cells within 7 cell radii. Please fix this by possibly defining a set which are those within 7 cell radii.

We have done this.

Also reminds me of how some people stick so stubbornly to "tradition! rah it's my tradition it doesn't matter!", yet we should be careful to not continue to use outdated traditions that harm others.

Author response:

The following is the authors’ response to the original reviews.

Public Reviews:

Reviewer #1 (Public Review):

Summary:

This study provides an incremental advance to the scavenger receptor field by reporting the crystal structures of the domains of SCARF1 that bind modified LDL such as oxidized LDL and acylated LDL. The crystal packing reveals a new interface for the homodimerization of SCARF1. The authors characterize SCARF1 binding to modified LDL using flow cytometry, ELISA, and fluorescent microscopy. They identify a positively charged surface on the structure that they predict will bind the LDLs, and they support this hypothesis with a number of mutant constructs in binding experiments.

Strengths:

The authors have crystallized domains of an understudied scavenger receptor and used the structure to identify a putative binding site for modified LDL particles. An especially interesting set of experiments is the SCARF1 and SCARF2 chimeras, where they confer binding of modified LDLs to SCARF2, a related protein that does not bind modified LDLs, and use show that the key residues in SCARF1 are not conserved in SCARF2.

Weaknesses:

While the data largely support the conclusions, the figures describing the structure are cursory and do not provide enough detail to interpret the model or quality of the experimental X-ray structure data. Additionally, many of the flow cytometry experiments lack negative controls for non-specific LDL staining and controls for cell surface expression of the SCARF constructs. In several cases, the authors interpret single data points as increased or decreased affinity, but these statements need dose-response analysis to support them. These deficiencies should be readily addressable by the authors in the revision.

The paper is a straightforward set of experiments that identify the likely binding site of modified LDL on SCARF1 but adds little in the way of explaining or predicting other binding interactions. That a positively charged surface on the protein could mediate binding to LDL particles is not particularly surprising. This paper would be of greater importance if the authors could explain the specificity of the binding of SCARF1 to the various lipoparticles that it does or does not bind. Incorporating these mutants into an assay for the biological role of SCARF1 would be powerful.

Reviewer #2 (Public Review):

Summary:

The manuscript by Wang and colleagues provided mechanistic insights into SCARF1 and its interactions with the lipoprotein ligands. The authors reported two crystal structures of the N-terminal fragments of SCARF1 ectodomain (ECD). On the basis of the structural analysis, the authors further investigated the interactions between SCARF1 and modified LDLs using cell-based assays and biochemical experiments. Together with the two structures and supporting data, this work provided new insights into the diverse mechanisms of scavenger receptors and especially the crucial role of SCARF1 in lipid metabolism.

Strengths:

The authors started by determining the crystal structures of two fragments of SCARF1 ECD. The superposition of the two high-resolution structures, together with the predicted model by AlphaFold, revealed that the ECD of SCARF1 adopts a long-curved conformation with multiple EGF-like domains arranged in tandem. Non-crystallographic and crystallographic two-fold symmetries were observed in crystals of f1 and f2 respectively, indicating the formation of SCARF1 homodimers. Structural analysis identified critical residues involved in dimerization, which were validated through mutational experiments. In addition, the authors conducted flow cytometry and confocal experiments to characterize cellular interactions of SCARF1 with lipoproteins. The results revealed the vital role of the 133-221aa region in the binding between SCARF1 and modified LDLs. Moreover, four arginine residues were identified as crucial for modified LDL recognition, highlighting the contribution of charge interactions in SCARF1-lipoprotein binding. The lipoprotein binding region is further validated by designing SCARF1/SCARF2 chimeric molecules. Interestingly, the interaction between SCARF1 and modified LDLs could be inhibited by teichoic acid, indicating potential overlap in or sharing of binding sites on SCARF1 ECD.

The author employed a nice collection of techniques, namely crystallographic, SEC, DLS, flow cytometry, ELISA, and confocal imaging. The experiments are technically sound and the results are clearly written, with a few concerns as outlined below. Overall, this research represents an advancement in the mechanistic investigation of SCARF1 and its interaction with ligands. The role of scavenger receptors is critical in lipid homeostasis, making this work of interest to the eLife readership.

Reviewer #3 (Public Review):

Summary:

The manuscript by Wang et. al. described the crystal structures of the N-terminal fragments of Scavenger receptor class F member 1 (SCARF1) ectodomains. SCARF1 recognizes modified LDLs, including acetylated LDL and oxidized LDL, and it plays an important role in both innate and adaptive immune responses. They characterized the dimerization of SCARF1 and the interaction of SCARF1 with modified lipoproteins by mutational and biochemical studies. The authors identified the critical residues for dimerization and demonstrated that SCARF1 may function as homodimers. They further characterized the interaction between SCARF1 and LDLs and identified the lipoprotein ligand recognition sites, the highly positively charged areas. Their data suggested that the teichoic acid inhibitors may interact with SCARF1 in the same areas as LDLs.

Strengths:

The crystal structures of SCARF1 were high quality. The authors performed extensive site-specific mutagenesis studies using soluble proteins for ELISA assays and surface-expressed proteins for flow cytometry.

Weaknesses:

(1) The schematic drawing of human SCARF1 and SCARF2 in Fig 1A did not show the differences between them. It would be useful to have a sequence alignment showing the polymorphic regions.

The schematic drawing in Fig.1A is to give a brief idea about the two molecules, the sequence alignment may take too much space in the figure. A careful alignment between SCARF1 and SCARF2 can be found in Ref. 24 (Ishii, et al., J Biol Chem, 2002. 277, 39696-702) an also mentioned in p.4.

(2) The description of structure determination was confusing. The f1 crystal structure was determined by SAD with Pt derivatives. Why did they need molecular replacement with a native data set? The f2 crystal structure was solved by molecular replacement using the structure of the f1 fragment. Why did they need to use EGF-like fragments predicted by AlphaFold as search models?

The crystal structure of f1 was first determined by SAD using Pt derivatives, but soaking of Pt reduced the resolution of the crystals, therefore we use this structure as a search model for a native data set that had higher resolution for further refinement. For the structural determination of f2, the molecular replacement using f1 structure was not able to show the initial density of the extra region in f2 (residues 133-209), which was missing in f1. Therefore, the EGF-like domains of SCARF1 modeled by AlphaFold were applied as search models for this region (p.18).

(3) It's interesting to observe that SCARA1 binds modified LDLs in a Ca2+-independent manner. The authors performed the binding assays between SCARF1 and modified LDLs in the presence of Ca2+ or EDTA on Page 9. However, EDTA is not an efficient Ca2+ chelator. The authors should have performed the binding assays in the presence of EGTA instead.

The binding assays in the presence of EGTA are included in the revised manuscript (Fig. S7) (p.9), which also suggest that SCARA1 binds OxLDL in a Ca2+-independent manner.

(4) The authors claimed that SCARF1Δ353-415, the deletion of a C-terminal region of the ectodomain, might change the conformation of the molecule and generate hinderance for the C-terminal regions. Why didn't SCARF1Δ222-353 have a similar effect? Could the deletion change the interaction between SCARF1 and the membrane? Is SCARF1Δ353-415 region hydrophobic?

The truncation mutants were constructed to roughly locate the binding region of lipoproteins on SCARF1, and the overall results showed that the sites might locate at the region of 133-221. Mutant Δ222-353 may also affect the conformation, but it still had binding with OxLDL like wild type, suggesting the binding sites were retained in this mutant. Mutant Δ353-415 showed a reduction of binding, implying that the binding sites might be retained but binding was affected, we think it might be due to the conformational change that could reduce the binding or accessibility of lipoproteins. Since this region locates closer to the membrane, it’s possible that it may change the interaction with the membrane. In the AF model, Δ353-415 region does not seem to be more hydrophobic than other regions (Fig. S2C).

(5) What was the point of having Figure 8? Showing the SCARF1 homodimers could form two types of dimers on the membrane surface proposed? The authors didn't have any data to support that.

Fig. 8 shows a potential model of the SCARF1 dimers on the cell surface by combining the structural information from crystals and AF predictions. The two dimers in the figure are identical but with different viewing angles. The lipoprotein binding sites are also indicated (Fig. 8).

Recommendations for the authors:

Reviewer #1 (Recommendations For The Authors):

The authors need to show examples of the electron density for both structures.

Electron density examples of the two structures are shown in Fig. S2A.

Figure 1)

The figure does not show enough details of the structure. The text mentions hydrogen-bond and disulfide bonds that stabilize the loops, these should be shown.

Disulfide bonds of the two structures are shown in Fig. 1.

Figure 2)

D) The full gel should be shown.

E) Rather than just relying on changes in gel filtration elution volumes, the authors do the appropriate experiment and measure the hydrodynamic radius of the WT and mutant ectodomains by DLS. However, they need to show plots of the size distribution, not just mean radial values, in order to show if the sample is monodisperse.

The full gel and plots of DLS are shown in Fig. S3A-B.

Figure 3)

I have concerns about the rigor of the experiments in panels A-D. The authors include a non-transfected control but do not appear to have treated non-transfected cells with the lipoproteins to evaluate the specificity of binding. Every cell binding assay (flow  or confocal) must show the data from non-transfected cells treated with each lipoprotein, as each lipoprotein species could have a unique non-specific binding pattern. The authors show these controls in Figure 6, but these controls are necessary in every experiment.

In Fig. 3A, since several lipoproteins were included in the figure, we use non-transfected cells without lipoprotein treatment as a negative control. The OxLDL or AcLDL treated non-transfected cells were also used as negative controls and shown in Fig. 3B-C. LDL, HDL or OxHDL may have their own non-specific binding patterns, the treatment of LDL, HDL or OxHDL with the transfected cells all gave negative results (Fig. 3A and D).

Cell-surface of the SCARF1 variants is a major concern. The constructs the authors use are tagged with a GFP on the cytosolic side. However, the Methods to do indicate if they gate on GFP+ transfected cells for analytical flow. Such gating may have been used because the staining experiments in Figures 3 and 4 show uniform cell populations, whereas the staining done with an anti-SCARF1 Ab in S4 shows most of the cells not expressing the protein on the surface. Please clarify.

Data for the anti-SCARF1 Ab assay is gated for GFP in the revised Fig. S4, and  the non-transfected cells are included as a control.

The authors must demonstrate cell-surface staining with an epitope tag on the extracellular side and clarify if the analyzed cells are gated for surface expression. The anti-SCARF antibody used in S4 may not recognize the truncated or mutant SCARFs equally. Cell-surface expression in the flow experiments cannot be inferred from confocal experiments because the flow experiments have a larger quantitative range.

Anti-SCARF1 antibody assay provides an estimation of the surface expression of the proteins. If the epitope of the antibody was mutated or removed in the mutants, most likely it would lose binding activity. Including an epitope tag on the ectodomain could be an option, but if truncation or mutation changes the conformation of the ectodomain, the accessibility of the epitope may also be affected, and addition of an extra sequence or domain, such as an epitope tag, may affect the surface expression of proteins sometimes.

In several places, the authors infer increased or decreased affinity from mean fluorescent intensity values of a single concentration point without doing appropriate dose-curves. These experiments need to be done or else the mentions of changes in apparent affinities should be removed.

We add a concentration for the WT interaction with OxLDL (Fig. S6, p.9) and the manuscript is also modified accordingly.

Figure 7

The concentration of teichoic acid used to inhibit modified LDL binding should be indicated and a dose-curve analysis should be done comparing teichoic acid to some non-inhibitory bacterial polymer.

The concentration of teichoic acids used in the inhibition assays is 100 mg/ml (p.21). Unfortunately, we don’t have other bacterial polymers in the lab and not sure about the potential inhibitory effects.

Reviewer #2 (Recommendations For The Authors):

Major points:

(1) The SCARF1 ECD contains three N-linked glycosylation sites (N289, N382, N393). It remains unclear whether these modifications are involved in SCARF1 binding to modified LDLs. Is it possible to design some experiments to investigate the effect of N-glycans on the recognition of modified LDLs? In particular, N382 and N393 are included in 353-415aa and the truncation mutant of SCARF1Δ353-415aa resulted in reduced binding with OxLDL in Fig.3G. Or whether the reduced binding is only due to the potential conformational changes caused by the deletion of the C-terminal region of the ECD?

A previous study regarding the N-glycans (N289, N382, N393) of SCARF1 (ref.17) has shown that they may affect the proteolytic resistance, ligand-binding affinity and subcellular localization of SCARF1, which is not quite surprising as lipoproteins are large particles, the N-glycans on the surface of SCARF1 could affect accessibility or affinity for lipoproteins. But the exact roles of each glycan could be difficult to clarify as they might also be involved in protein folding and trafficking.

The reduction of the binding of OxLDL for the mutant SCARF1 Δ353-415aa may be due to the conformational change or the loss of the glycans or both.

(2) The authors speculated that the dimeric form of SCARF1 may be more efficient in recognizing lipoproteins on the cell surface. Please highlight the critical region/sites for ligand binding in Figure 8 and discuss the structural basis of dimerization improving the binding.

The binding sites for lipoproteins on SCARF1 are indicated in Fig. 8. According to our data, it might be possible the conformation of the dimeric form of SCARF1 makes it more accessible to the ligands on the cell surface as implied by flow cytometry (p.14-15), but still needs further evidence on this.

(3) Could the two salt bridges (D61-K71, R76-D98) observed in f1 crystals be found in f2 crystals? They seemed to be a little far from the defined dimeric interface (F82, S88, Y94) and how important are these to SCARF1 dimerization?

The two salt bridges observed in f1 crystal are not found in f2 crystal (distances are larger than 5.0 Å), suggesting they are not required for dimerization (p. 7-8), but may be helpful in some cases.

(4) The monomeric mutants (S88A/Y94A, F82A/S88A/Y94A) exhibited opposite affinity trends to OxLDL in ELISA and flow cytometry. The authors proposed steric hinderance of the dimers coated onto the plates as the potential explanation for this observation. However, the method of ELISA stated that OxLDLs, instead of SCARF1 ECD, were coated onto the plates. So what's the underlying reason for the inconsistency in different assays?

Thanks. ELISA was done by coating OxLDLs on the plates as described in the Methods. But still, a dimeric form of SCARF1 may only bind one OxLDL coated on the plates due to steric hinderance. We correct this on p.12.

Minor points:

(1) Figure 2D and Figure S3 - please label the molecular weight marker on the SEC traces to indicate the native size of various purified proteins.

The elution volume of SEC not only reflects the molecular weight, but it’s also affected by the conformation or shape of protein. The ectodomain of SCARF1 has a long curved conformation, the elution volumes of the monomeric or dimeric forms of SCARF1 do not align well with the standard molecular weight marker and elute much earlier in SEC. We include the standard molecular weight marker in Fig. S3C-D.

(2) Could the authors provide SEC profiles of f1 and f2 that were used in crystallographic study?

The SEC profiles of f1 and f2 for crystallization are shown in Fig. S5 (p.6).

(3) The legend of Figure 3A states that the NC in flow cytometry assay represents the non-transfected cells, but please confirm whether the NC in Fig. 3A-C corresponds to non-transfected cells or no lipoprotein.

NC in Fig. 3A represents the non-transfected cells, and no lipoproteins were added in this case as several lipoproteins are included in Fig. 3A. The lipoprotein (OxLDL or AcLDL) treated non-transfected cells (NC) were shown in Fig. 3B-C as negative controls.

We see our lives as non fiction, as real but we question that as there’s nothing o back that up. I really like how Plato speaks on these reading because it shows how easy it is to change someone’s belifs and perspective, wethers it’s religion or just general beliefs

This part talks about the practical side of philosophy. Sure, it teaches you how to think and write better, which is helpful for just about any job. But the writer points out that this view is a bit underwhelming. Philosophy isn’t just about being useful—it’s about shaking things up and helping you see the world differently. It’s kind of like saying philosophy isn’t just a tool; it’s a way of life.

Constructive reimagination is a reminder to us that philosophy is not just about critque, it's about building a better world, helping one imagine a future that's more just and empowering.

This line is saying that people have started to care more about using science to do and create things, rather than just learning about science for it's own sake. There is a growing interest in the practical stuff science can do.

I think that human language being comprised of not only verbal components but physical ones aswell is what makes people so intruiging. It makes me think about communication between animals, insects, and even fungi. I love that there are so many intricate ways one can communicate depending on who it's to, the time, place, and information being communicated. Language being multi-faceted is one of the things I feel helps people learn how to think critically when they're growing up. You learn to pick up on cues and the 'right' ways to respond to certain information and situations. As a kid, I remember blurting things out that, while true, had negative impacts on people around me at the time. I didn't understand why they were upset because, in my mind, I was just relaying information. Figuring out that there is a time and place for certain things is important, or better yet not saying anything at all.

This vagueness of life vs. death feels similar to the vagueness of Hyacinth's gender from a few lines up. Historically, Hyacinthus was a male god beloved by Apollo who was tragically killed. A yearly festival called Hyacinthia was created in his honor. It's interesting that the festival was called Hyacinthia because the -a ending is normally more associated with femininity and female names. In the poem, Eliot uses the term "hyacinth," as if he's trying to assign no gender to what could have been Hyacinthus/ia. This intentional vagueness connects to the line "I was neither / Living nor dead" (Eliot, 39-40) because Hyacinth is neither male nor female just like the narrator was "neither living nor dead". Essentially, these things all cancel each other out.

This idea differs from many of the lines presented in Mina Loy's poem "At the Door of the House". She refers to eyes being "riveted to the unrealisable" (Loy 1). The verb and noun here essentially cancel each other out with the impossibility of the situation. However, this differs to the idea presented in the first paragraph because these subjects aren't both rejected by the author. Hyacinth can't have a gender, the narrator can't be dead or alive, but the eyes are still "riveted to the unrealisable". The event presented here that inherently contradicts itself doesn't seem to be rejected, unlike how Eliot does it with life and death and with Hyacinth's gender.

In “The Fire Sermon” one of Eliot’s passages poses an interesting contrast to a passage from one of his sources of inspiration “Towards Democracy” by John Heywood. This contrast exposes the deception of industrialist and nationalist hopes and the subsequent and melancholy reality of such progressions. On lines 191-197, Eliot states, “Musing upon the king my brother’s wreck//And on the king my father’s death before him//White bodies naked on the low damp ground//And bones cast in a little low dry garret//Rattled by the rat’s foot only, year to year//But at my back from time to time I hear//The sound of horns and motors”. The mention of brothers and fathers comments on nationalist sense of unity between men, however, in a negative sense, as Eliot mentions “wreck”, and “death” in relation to these familial figures. The “King” might also refer to the Fisher King, a mythological figure previously mentioned in the poem, whose incurable wounds have left him in a constant and repetitive state of fishing. Once tasked with noble duties such as guarding the holy grail, he is left with his wounds. What some might consider an inescapable fall from grace also plays into Eliot’s regretful reflection of his “brother” and “father”. Eliot describes bodies on the “low” ground, as well as bones in a garret. These dead things are only disturbed by the rats, which are often percieved as carries of disease and signifiers of abandonment and decay. Despite the narrator’s bleak description, they also regard the noises horns and motors in an opposite light, perhaps in search of some appreciation for industrialism. Also likely is that the narrator is falling into industrialist narratives of progress once again, forgetting the desolation they had just described. Heywood’s passage seems to align itself with these industrialist narratives and hopes for progress. He states, “I see a great land poised as in a dream Waiting for the word by which it may live again. I see the stretched sleeping figure waiting for the kiss and the re-awakening. I hear the bells pealing, and the crash of hammers, and see beautiful parks spread as in toy show. I see a great land waiting for its own people to come and take possession of it. Heywood’s description is not immediately positive- in fact, there seems to be anticipation and perhaps even a restlessness of waiting for the reclamation of the land. In this waiting state, Heywood describes the land as a dream, and something not living. The sleeping figures described by Heywood also evoke Eliot’s descriptions of the naked white bodies. Interestingly, Heywood’s description of noises of production as positive, describing sparks as beautiful, which relates to the narrator of “The Fire Sermon” appreciating the sounds of horns and motors”. However, perhaps Heywood compliments these noises in a mocking manner, as he describes them as a toy show. Heywood ends his passage calling for the reclamation of the land by the people who live on it, creating a relationship of dominance towards the land typical of industrialist ideas. Ultimately, Eliot’s and Heywood’s views of the land display the hopes and impatience of human progress and it’s later and sometimes devastating effects.

In the first part of The Game of Chess, the female character is isolated and defined by the lifeless objects surrounding her—perfumes, glass, candle flames, lacquer, and more. Only through these objects, do we get a chance to become acquainted with her. These inanimate items symbolize suffocation and entrapment in her loneliness. The readers can only speculate if it’s the notion of rape that Eliot continuoulsy references that led to this isolation. No matter the cause of it, however, the character strives to get herself out of this situation. She strives for human connection. Her plea, “Speak to me,” reflects this need, but the absence of a question mark in “Why do you never speak to me” suggests that the character already knows the answer. The reader, however, is left to speculate: does she see herself as undesirable because of her trauma? Or is it simply the years of a relationship that deteriorate this connection referencing Eliot’s own troubled marriage? In either case, this emotional disconnect is further demonstrated by the subsequent question, “What are you thinking of?” This time, the question is marked by a question mark, suggesting an actual attempt to break through this emotional barrier. These attempts, however, are ineffective, as the desperation rises and the questions shorten to “what thinking” and “what.” This fragmented monologue mirrors the fragmentation of her emotional state.

In contrast, the second character suffers from the destructive excess of human connection. Shamed for her appearance, she faces the reality of her partner’s potential infidelity, reflected in the statement, “And if you don’t give it to him, there’s others.” The references to abortion intensify this degradation. She justifies her loss of beauty and confidence with the line, “It’s them pills I took, to bring it off.” Instead of finding fulfillment in connection, this character’s relationships strip her of her self-worth.

In these two cases, Eliot presents women trapped at the opposite extremes of human connection: one suffers from its absence, the other from its destructive abundance. Yet, in both cases, external forces define and entrap them. The first woman is reduced to the objects around her, while the second is judged by an external voice—the pronoun “I” suggesting our, as readers, own judgment projected onto the character. We become not simply the judges, but also the victimes of this broken connection. The poem’s fragmented language, which severely affects our understanding of it, mirrors the emotional chaos, invoking feelings rather than rationality, similarly to Ophelia’s “mad” song in Hamlet.

This pattern mirrors the nature of chess, where a single wrong move can drastically alter the entire game. Just as in chess, life’s unpredictability is highlighted in these women’s lives, as one extreme of human connection can quickly shift to another, with equally devastating outcomes. The title of the section, The Game of Chess, thus, reflects this instability —one wrong move leads to extremes of connection. In this case, these characters and we, as readers, are not simply entrapped in this game. Instead, we are playing with an unwinnable position from the very outset: each move only brings the inevitability of loss closer.

The first few times I read this passage, I glossed over the spelling of “Goonight”. It wasn’t until the indented Ophelia soliloquy “Good night, ladies, good night, sweet ladies, good night, / good night.” that I noticed that “Goonight” was missing a “d” and a space. I attributed this to the slurrish / slangy vernacular of the unnamed first person subject. The portmanteauing of the word evokes a sense of urgency, as if there isn’t enough time for the character to pronounce two full syllables separately and must combine the two into a breath. This aligns with the repeated phrases “HURRY UP PLEASE ITS TIME”, which as I am typing this, I realize is also missing punctuation “ ‘ “ that critically indicates that the word is a contraction. Again, a word is shortened, this time dropping even the apostrophe. “ITS” is also a particularly interesting case because “IT’S” represents contraction “IT IS”, which is grammatically correct and therefore what I assume the correct voice is, while just “ITS” is a possesive pronoun modifying who knows what. Whose time is it? Who does time belong to? Do we all belong to time?

The imperative “HURRY UP ITS TIME” is repeated four times in under 40 lines, first spaced out, and then condensing to repeat back to back 4 lines before the end of the poem. This increasingly urgent demand phonetically (and emotionally) elicits the pressures of living. As a third person / exterior overhead voice in the scene between Lil and the first person subject consoling/incriminating her of her domestic shortcomings as a mother and wife, the phrase is an objective pace keeper, separate from the cast of characters and their woes. Every time Lil is about to answer an emotionally wrought question, (“What you get married for if you don’t want children?”), the phrase in all-caps becomes her saving grace, interrupting the accusatory line of questioning between one character (a condescending wife or a oatrarchy-upholding man?) and her truth. Outside the scene, the phrase is a speed sign for us readers as we parse through the different voices. The insertion reads like Ophelia’s “sings” cues in Hamlet, where the theatrical cue physically separates delivered lines from those sung. Both interpretations embody the accelerando of life. Lil, her snobby friend, the various ambiguous pronouns, me, you, we are all of us running out of time, being pursued, pursued.