Details about Chromecast's security:

security after it's setup

Anyone who can connect to your wifi network can send video to your Chromecast. As long as your wifi network is password-protected, your Chromecast is secure. [1] [2]

Well, except for one thing:

security when it's first setup

While it's being first setup, it's vulnerable to being hijacked by someone else. After setup, it's secure.

But it's not in setup mode for very long, so everything is fine, right?

Problem — there's an easy way for an attacker to put it in setup mode, after which they can instruct it to connect to the attacker's network. After that, they can Rickroll your TV all day long.