Docker is not designed to work that way. It is considered a bad practice to be able to ssh into a running container.
First of all realize there are two things of concern, Images and Containers. Think of the Image as the contents of a hard drive. That is where you make changes you want to persist. But just like a hard drive, Images are inert.
A Container is like a program that has been loaded into memory and is executing. A Container is ephemeral. And you can have more than one Container running from the same Image.
Now a completely ephemeral container is not much good to anyone. This is what Volumes are for. A volume is a way to mount a folder from the host into the container. This is where you put your configs and databases and such. Because the data gets saved to the host, it gets persisted even when the Image changes and new Containers are created. So, for openHAB 2 one mounts the conf and user data folders into the container. For Mosquito, one mounts /etc/mosquito into the container.
There might be times when you want to make a change to a Container, for example when debugging a problem or adding new features, but your changes will not persist unless you create a new Image from that container, or even better, modify the Dockerfile to make the changes to the Image that gets built.
You can do this by running:
docker exec mosquito /bin/bash
Assuming the name of your container is “mosquito”.
By, like I said above, the proper way to configure mosquito would be to mount a volume to conf. Then you edit the mqtt.cfg file on the host.
Please see the basic tutorial on Docker’s website, review the op above again, paying attention to where it talks about volumes. And review the readme for the openHAB Docker Image on dockerhub and/or the GitHub repo for the docker image.