Mosquitto can't change password instructions not correct

[smithy-eng]

Hi

1. In your IOT docs you have a video link to change password, bu youtube video says 'video unavailable'
2. I have tried changing user from "1883" to "0" in the docker compose.yml file but it does not work, also the text you ask me to find is actually not correct, the .yaml file has different text - please check, user stays as 1883
3. I added user root mosquitto.conf but this makes no difference
4. /bin/bash does not work from console but no instructions on this?
5. restart container/reboot complete pi no difference no access mosquito user reports logged in as 1883

Thanks
C

[Paraphraser]

Starting position:

1. Set working directory:

   $ cd ~/IOTstack
   

2. ./docker-compose.yml fragment for mosquitto:

     mosquitto:
       container_name: mosquitto
       image: eclipse-mosquitto
       restart: unless-stopped
       user: "1883"
       ports:
         - "1883:1883"
       volumes:
         - ./volumes/mosquitto/data:/mosquitto/data
         - ./volumes/mosquitto/log:/mosquitto/log
         - ./volumes/mosquitto/pwfile:/mosquitto/pwfile
         - ./services/mosquitto:/mosquitto/config:ro
        networks:
         - iotstack_nw
   

   Yes, I know this is slightly different to the template. See Issue 269 for the explanation.

   Later edit to add the last two lines - needed if you are running "new menu".

3. contents of ./services/mosquitto/mosquitto.conf

   listener 1883
   
   persistence true
   persistence_location /mosquitto/data
   
   #log_dest file /mosquitto/log/mosquitto.log
   log_dest stdout
   
   #password_file /mosquitto/pwfile/pwfile
   allow_anonymous true
   
   #acl_file /mosquitto/config/filter.acl
   
   log_timestamp_format %Y-%m-%dT%H:%M:%S
   

   Yes, I know the pwfile is disabled at this point. Trust me!

   There are some other subtle differences too (grouping related elements, removing the trailing slash from the persistence location) but those are cosmetic rather than material changes.

4. if mosquitto is running, terminate it:

   $ docker-compose stop mosquitto
   $ docker-compose rm -f mosquitto
   

5. if there is an existing password file, get rid of it:

   $ sudo rm ./volumes/mosquitto/pwfile/pwfile
   

6. make sure permissions are correct in the persistent storage area:

   $ sudo chown -R 1883:1883 ./volumes/mosquitto
   

   This is normally done by ./.templates/mosquitto/directoryfix.sh.

7. make sure mosquitto is up-to-date (should be ≥2.0.7 - see step 9 below)

   $ docker-compose pull mosquitto
   

8. bring up mosquitto:

   $ docker-compose up -d mosquitto
   

9. check the log to make sure it looks happy (you might need to wait a while to get more than one line):

   $ docker logs mosquitto
   2021-02-16T03:22:56: mosquitto version 2.0.7 starting
   2021-02-16T03:22:56: Config loaded from /mosquitto/config/mosquitto.conf.
   2021-02-16T03:22:56: Opening ipv4 listen socket on port 1883.
   2021-02-16T03:22:56: Opening ipv6 listen socket on port 1883.
   2021-02-16T03:22:56: mosquitto version 2.0.7 running
   

10. create the FIRST password like this:

    $ docker exec mosquitto mosquitto_passwd -c -b /mosquitto/pwfile/pwfile «username» «password» 
    

    Replace "«username»" and "«password»" as appropriate, for example:

    $ docker exec mosquitto mosquitto_passwd -c -b /mosquitto/pwfile/pwfile hello world 
    

11. create SECOND and SUBSEQUENT passwords like this:

    $ docker exec mosquitto mosquitto_passwd -b /mosquitto/pwfile/pwfile «username» «password» 
    

    The key difference is the omission of the -c flag (means "create the password file").

    Example:

    $ docker exec mosquitto mosquitto_passwd -b /mosquitto/pwfile/pwfile theworld isnotenough
    

12. check that the password file exists

    $ cat ./volumes/mosquitto/pwfile/pwfile 
    hello:$7$101$rGEPZWRc6q8wX6KW$FRwyoemymo5RMUE71uTXVy2Oqfwl5aDf2VJNBgwCQtK7gqH2LSnn4OHKVG3FIRF15nO8AGwhZPbJq6jK2m7XPQ==
    theworld:$7$101$fHV4JIZwqkF79gAa$3blXsRd4zO9vUoTEpCLXgVeP/I8tMty21j27nXXzK4kd8GLuwJvsi2kTIQFZ4E2ttF5fuRAHEoxPGxvUw50lSA==
    

13. edit ./services/mosquitto/mosquitto.conf to be like this

    listener 1883
    
    persistence true
    persistence_location /mosquitto/data
    
    #log_dest file /mosquitto/log/mosquitto.log
    log_dest stdout
    
    password_file /mosquitto/pwfile/pwfile
    allow_anonymous false
    
    #acl_file /mosquitto/config/filter.acl
    
    log_timestamp_format %Y-%m-%dT%H:%M:%S
    

    The changes are to enable the pwfile and turn off anonymous access.

14. restart mosquitto (so it re-reads the config file)

    $ docker-compose restart mosquitto
    

15. assuming sudo apt install mosquitto-clients, test without credentials

    $ mosquitto_pub -h 127.0.0.1 -p 1883 -t "/password/test" -m "up up and away"
    Connection Refused: not authorised.
    Error: The connection was refused.
    

16. test with credentials

    $ mosquitto_pub -h 127.0.0.1 -p 1883 -t "/password/test" -m "up up and away" -u hello -P world
    $ 
    

    Silence = success!

17. prove round-trip connectivity by setting up a subscriber as a background process:

    $ mosquitto_sub -v -h 127.0.0.1 -p 1883 -t "/password/test" -F "%I %t %p" -u theworld -P isnotenough &
    [1] 25996
    

    Repeat the test

    $ mosquitto_pub -h 127.0.0.1 -p 1883 -t "/password/test" -m "up up and away" -u hello -P world
    2021-02-16T14:40:51+1100 /password/test up up and away
    

    the second line is coming from the background process.

    Kill the background process when you're done

    $ kill %1
    [1]+  Terminated              mosquitto_sub -v -h 127.0.0.1 -p 1883 -t "/password/test" -F "%I %t %p" -u theworld -P isnotenough
    

---

Although I've just shown it to be working, you're not alone in scratching your head about this whole topic (no pun intended).

See also this gist. I started out with nothing working, and now I can't replicate the fault.

It confuses the heck out of me.

---

If you think the doco needs to be improved then the quickest way to make it happen is to prepare a Pull Request.

There's a guide at this gist.

Hope this helps!

[Londoneye02]

Thank you for the thorough explanation !!!

I was about to open an Issue myself regarding this...

this part made the difference for me: $ sudo chown -R 1883:1883 ./volumes/mosquitto

somehow this was not automatically done :-)

Thank you again

[Paraphraser]

@Londoneye02 if you want an in-depth discussion of the problems with Mosquitto, I've opened this issue on their repo. You'll notice there's not a whole lot of response and I don't know what to make of that. Maybe I gave them too much info.

In theory IOTstack "solves" this problem by running directoryfix.sh. Under old menu this was copied from the .templates directory to the services directory and executed there towards the end of the menu run. The script is still in the new menu template but I'm beginning to suspect it isn't being run by the new menu. However, if it is being run by both menus then it follows that it not being run on your system is some difference on your system which either prevented it from being run or prevented it from taking effect. I can't speculate as to which.

As you'll see from the issue I referenced above, I think the presence of a directoryfix.sh is pretty much the signature of a non-well-behaved container. Containers that are well-behaved self repair. NodeRed is a good example. If you erase its directory in volumes then bring up the stack, docker-compose will create everything defined in its volumes list (as empty directories) and then NodeRed notices the empty directories and initialises with sensible defaults. I'd like to try to get Mosquitto to the same point so that problems like this never trap anyone.

[Paraphraser]

Well and truly out of date. Self-repair guarantees null password file on first launch or if the file ever goes walkabout. Doco explains add/delete passwords.