Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.
According to the Rights Lens, the AI companies wouldn't have the right to expose the public to potentially harmful experiments. If at any point one person doesn’t want to participate in this, they are having their own rights violated, and therefore the company shouldn't do it. Through this lens, the line between being a means to an end and not, is drawn almost immediately. This would be especially true if it meant people forfeiting their right for the sake of the company and what they are making. When we look at this through a Utilitarian Lens, things change a little bit. If the work of the AI companies results in more people benefiting than without them developing it, it would be ethical. In the utilitarian lens, they would look to see who and how the companies are affecting the people. If the product that the company would invent would prevent generational hurt and damage, I would argue through this lens, it's worth the potential ‘bad’. It would start treating people as a means rather than an end if the end result was no longer needed, or it was more costly to produce the potential benefits.