The Authentication and Authorization for Constrained Environments (ACE) framework provides fine-grained access control in the Internet of Things, where devices are resource-constrained and with limited connectivity. The ACE framework defines separate profiles to specify how exactly entities interact and what security and communication protocols to use. This paper presents the novel ACE IPsec profile, which specifies how a client establishes a secure IPsec channel with a resource server, contextually using the ACE framework to enforce authorized access to remote resources. The profile makes it possible to establish IPsec Security Associations, either through their direct provisioning or through the standard IKEv2 protocol. We provide the first Open Source implementation of the ACE IPsec profile for the Contiki OS and test it on the resource-constrained Zolertia Firefly platform. Our experimental performance evaluation confirms that the IPsec profile and its operating modes are affordable and deployable also on constrained IoT platforms.