Interactive storiesStorybook enables you to capture various states of a component in a structured format called a story. Each story simulates a particular UI state by rendering the component in isolation and supplying props and mock data.However, some UI states can only be reached via user interaction—click, drag, tap, type, etc. These were previously impossible to model in Storybook. To address this, we've added a new construct in Storybook 6.4 called the play function. That enables you to run scripted interactions after a story is rendered.Consider this play function that fills out a form to trigger a validation:

You can think of a test as being broken down into four steps: Arrange Act Assert Cleanup Arrange is where we prepare everything for our test. This means pretty much everything except for the “act”. It’s lining up the dominoes so that the act can do its thing in one, state-changing step. This can mean preparing objects, starting/killing services, entering records into a database, or even things like defining a URL to query, generating some credentials for a user that doesn’t exist yet, or just waiting for some process to finish. Act is the singular, state-changing action that kicks off the behavior we want to test. This behavior is what carries out the changing of the state of the system under test (SUT), and it’s the resulting changed state that we can look at to make a judgement about the behavior. This typically takes the form of a function/method call. Assert is where we look at that resulting state and check if it looks how we’d expect after the dust has settled. It’s where we gather evidence to say the behavior does or does not align with what we expect. The assert in our test is where we take that measurement/observation and apply our judgement to it. If something should be green, we’d say assert thing == "green". Cleanup is where the test picks up after itself, so other tests aren’t being accidentally influenced by it. At it’s core, the test is ultimately the act and assert steps, with the arrange step only providing the context. Behavior exists between act and assert.

But it is kind of verbose, and it’s also a lot slower, so I can understand why the model test might be preferred.

I am firmly convinced that asserting on the state of the interface is in every way superior to asserting on the state of your model objects in a full-stack test.

I suggest renaming this to something like SomeInterfaceAsTypeWrittenByHand. Because one of the reasons for Simplify is so you don't have to write SomeType by hand. Then add the following to show that Simplify<SomeInterface> is the same as SomeInterfaceAsTypeWrittenByHand. declare const a: Simplify<SomeInterface>; expectType<SomeInterfaceAsTypeWrittenByHand>(a); // Interface is assignable to its Simplified type (created with Simplify, and by hand) expectType<Simplify<SomeInterface>>(someInterface); expectType<SomeInterfaceAsTypeWrittenByHand>(someInterface);

though we recommend Jest as our preference, the library works with any framework

The more your tests resemble the way your software is used, the more confidence they can give you.

NEW SECTION: What is Money?

Fixtures are created when first requested by a test, and are destroyed based on their scope: function: the default scope, the fixture is destroyed at the end of the test.

When pytest goes to run a test, it looks at the parameters in that test function’s signature, and then searches for fixtures that have the same names as those parameters. Once pytest finds them, it runs those fixtures, captures what they returned (if anything), and passes those objects into the test function as arguments.

“Fixtures”, in the literal sense, are each of the arrange steps and data. They’re everything that test needs to do its thing.

Testing at GitLab is a first class citizen, not an afterthought. It’s important we consider the design of our tests as we do the design of our features.

It’s fun but when would we ever use things like this in actual code?When it’s well tested, commented, documented, and becomes an understood idiom of your code base.We focus so much on black magic and avoiding it that we rarely have a chance to enjoy any of the benefits. When used responsibly and when necessary, it gives a lot of power and expressiveness.

SIT 是系統開發上的專有名詞 System integration testing 的縮寫

A lot of projects leveraging CDP appeared since then, including the most well-known one—Puppeteer, a browser automation library for Node.js. What about the Ruby world? Ferrum, a CDP library for Ruby, although being a pretty young one, provides a comparable to Puppeteer experience. And, what’s more important for us, it ships with a companion project called Cuprite—a pure Ruby Capybara driver using CDP.

That’s not the only way of writing end-to-end tests in Rails. For example, you can use Cypress JS framework and IDE. The only reason stopping me from trying this approach is the lack of multiple sessions support, which is required for testing real-time applications (i.e., those with AnyCable 😉).

Thus, by adding system tests, we increase the maintenance costs for development and CI environments and introduce potential points of failures or instability: due to the complex setup, flakiness is the most common problem with end-to-end testing. And most of this flakiness comes from communication with a browser.

For example, Database Cleaner for a long time was a must-have add-on: we couldn’t use transactions to automatically rollback the database state, because each thread used its own connection; we had to use TRUNCATE ... or DELETE FROM ... for each table instead, which is much slower. We solved this problem by using a shared connection in all threads (via the TestProf extension). Rails 5.1 was released with a similar functionality out-of-the-box.

In practice, we usually also need another tool to provide an API to control the browser (e.g., ChromeDriver).

“System tests” is a common naming for automated end-to-end tests in the Rails world. Before Rails adopted this name, we used such variations as feature tests, browser tests

even acceptance tests (though the latter are ideologically different)

Cuprite is a pure Ruby driver (read as no Selenium/WebDriver/ChromeDriver dependency) for Capybara.

This is why for a recent Angular+Rails project we chose to use a testing stack from the backend technology’s ecosystem for e2e testing.

We were not strictly blackbox testing our application. We wanted to simulate a user walking thru specific scenarios in the app which required that we have corresponding data in the database. This helps ensure integration between the frontend and backend was wired up successfully and would give us a foundation for testing critical user flows.

There are times to stretch individually and as a team, but there are also times to take advantage of what you already know.

When it came to testing the whole product, end-to-end, owning both sides gave us not only more options to consider, but also more tools to choose from.

This meant that we owned both sides of the product implementation. For unit testing on the frontend, we stayed with Angular’s suggestion of Jasmine. For unit testing on the backend, we went with rspec-rails. These worked well since unit tests don’t need to cross technology boundaries.

We used testing tools that were in the same ecosystem as our backend technology stack for primrily three reasons: We owned both ends of the stack Team experience Interacting with the database

This poses a few problems for automation. In some environments, there may be no graphical display available, or it may be desirable to not have the browser appear at all when being controlled.

Mocking is a form of global state like others (including ENV sharing), which will cause difficulties here (more with threads, a bit less with forks).

I’m going to add the API Server as an actor to my first test sequence to give some granularity as to what I’m actually testing.

For features like websocket interactions, a single full-stack smoke test is almost essential to confirm that things are going as planned, even if the individual parts of the interaction are also covered by unit tests.

Why does test suite performance matter? First of all, testing is a part of a developer's feedback loop (see @searls talk) and, secondly, it is a part of a deployment cycle.

How to test at the correct level?

As many things in life, deciding what to test at each level of testing is a trade-off:

Unit tests are usually cheap, and you should consider them like the basement of your house

A system test is often better than an integration test that is stubbing a lot of internals.

Only test the happy path, but make sure to add a test case for any regression that couldn’t have been caught at lower levels with better tests (for example, if a regression is found, regression tests should be added at the lowest level possible).

White-box tests at the system level (formerly known as System / Feature tests)

GitLab is transitioning from controller specs to request specs.

These kind of tests ensure that individual parts of the application work well together, without the overhead of the actual app environment (i.e. the browser). These tests should assert at the request/response level: status code, headers, body. They’re useful to test permissions, redirections, what view is rendered etc.

These tests should be isolated as much as possible. For example, model methods that don’t do anything with the database shouldn’t need a DB record. Classes that don’t need database records should use stubs/doubles as much as possible.

Black-box tests at the system level (aka end-to-end or QA tests)

White-box tests at the system level (aka system or feature tests)

Levels

White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) is a method of software testing that tests internal structures or workings of an application, as opposed to its functionality (i.e. black-box testing)