In The Avengers, Loki lost the sceptre belonging to Thanos when he failed to take over Earth. And going into Avengers: Age of Ultron, it's now in the possession of HYDRA's Baron von Strucker.

The Other, now dead at the hands of Ronan as we saw in Guardians of the Galaxy, warned Loki (in The Avengers) that there would be consequences if he failed his master.