Log4j vulnerability

Most probably they got rejected by the Nginx because of invalid characters or so, but I couldn’t find any logging for that.
And the fact that if it got somehow redirected to OH cloud it then (hopefully) only gets passed to the OH instance if someone has the correct credentials makes me feel a bit better :grin:
Thanks for your replies @J-N-K, @Flole and @Wolfgang_S

Apparently according to Log4j – Apache Log4j Security Vulnerabilities enabling log4j2.formatMsgNoLookups is not deemed sufficient enough…

The reason these measures are insufficient is that, in addition to the Thread Context attack vector mentioned above, there are still code paths in Log4j where message lookups could occur: known examples are applications that use Logger.printf("%s", userInput), or applications that use a custom message factory, where the resulting messages do not implement StringBuilderFormattable. There may be other attack vectors.

Wondering is openHAB is affected?

The new advice (apart from updating to log4j 2.16) is to remove org/apache/logging/log4j/core/lookup/JndiLookup.class from the JAR.

zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

For OH 2.5.12?
Is Pax logging 1.11.11 the version which is equivalent to 2.16.0? Which means the fixed version.

It is a little bit confusing that the numbers are not the same
Pax logging 2.0.12 = 2.16.0
Pax logging 1.1.11 = 2.16.0 ?

This will not solve your problem because you modify JAR in system/ directory (this is what I find from zip command), which is a “reference” location. At runtime pax logging is loaded from cached directory under data/cache. You still need to do an update call to force re-loading of module and refreshing of cache directory contents.

Your finding is correct. Pax logging does wrap log4j2 which causes us to track two versions at once. Usually it doesn’t matter, cause you get that from distro, yet this time it is important to know how to apply security. Have a look on my earlier answer Log4j vulnerability - #49 by splatch. This applies to any version of runtime and will work with 2.6.x too, despite of no patch release for it.

There is already a PR open on GitHub and the Karaf mailing list is watched closely. At Karaf there’s currently a vote for the release going on which will stay open for a little longer, the hope is that it will be closed early so Karaf will be released early. Then it could very much happen that it could be ready just in time.

3 Likes

If you ask me, the proper fix for this vulnerability does not require openhab to wait for Karaf release. Solution is here: Override pax logging version to address #1349. by splatch · Pull Request #1350 · openhab/openhab-distro · GitHub. It forces use of newest release of pax logging which contains fixed version of log4j.

Note old version of pax is still installed in filesystem, it can be removed later on as its early banishment causes build to fail.

1 Like

The 3.2.0 release includes 2.17.0, so it looks like this is fixed! Thanks everyone for your hard work addressing this so quickly

Hi.

How do i fix my OH 2 to use the patched pax files?

I manually put the new ones 1.11.13 in the openhab/runtime/system/org/ops4j/pax/ dir and removed the old ones…

I changed startup.properties to be …
mvn:org.apache.karaf.features/org.apache.karaf.features.extension/4.2.7 = 1
mvn:org.apache.felix/org.apache.felix.metatype/1.2.2 = 5
mvn:org.apache.karaf.services/org.apache.karaf.services.eventadmin/4.2.7 = 5
mvn:org.ops4j.pax.url/pax-url-aether/2.6.1 = 5
mvn:org.fusesource.jansi/jansi/1.18 = 8
mvn:org.ops4j.pax.logging/pax-logging-api/1.11.13 = 8
mvn:org.ops4j.pax.logging/pax-logging-log4j2/1.11.13 = 8
mvn:org.apache.felix/org.apache.felix.coordinator/1.0.2 = 9
mvn:org.apache.felix/org.apache.felix.configadmin/1.9.16 = 10
mvn:org.apache.felix/org.apache.felix.fileinstall/3.6.4 = 11
mvn:org.apache.karaf.features/org.apache.karaf.features.core/4.2.7 = 15

I removed the cache and restarted but seeing this is logs…

    Caused by: java.io.IOException: Error resolving artifact org.ops4j.pax.logging:pax-logging-api:jar:1.11.2: [Could not find artifact org.ops4j.pax.logging:pax-logging-api:jar:1.11.2 in openhab (https://openhab.jfrog.io/openhab/libs-release/)]
            at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.configureIOException(AetherBasedResolver.java:803) ~[?:?]
            at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:774) ~[?:?]
            at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:657) ~[?:?]
            at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:598) ~[?:?]
            at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:565) ~[?:?]
            at org.apache.karaf.features.internal.download.impl.MavenDownloadTask.download(MavenDownloadTask.java:52) ~[?:?]
            at org.apache.karaf.features.internal.download.impl.AbstractRetryableDownloadTask.run(AbstractRetryableDownloadTask.java:60) ~[?:?]
            ... 6 more
            Suppressed: shaded.org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.ops4j.pax.logging:pax-logging-api:jar:1.11.2 in openhab (https://openhab.jfrog.io/openhab/libs-release/)
                    at shaded.org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:48) ~[?:?]
                    at shaded.org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:368) ~[?:?]
                    at shaded.org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75) ~[?:?]
                    at shaded.org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:642) ~[?:?]
                    at shaded.org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:262) ~[?:?]
                    at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:489) ~[?:?]
                    at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:390) ~[?:?]
                    at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:215) ~[?:?]
                    at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:192) ~[?:?]
                    at shaded.org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact(DefaultRepositorySystem.java:247) ~[?:?]
                    at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:767) ~[?:?]
                    at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:657) ~[?:?]
                    at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:598) ~[?:?]
                    at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:565) ~[?:?]
                    at org.apache.karaf.features.internal.download.impl.MavenDownloadTask.download(MavenDownloadTask.java:52) ~[?:?]
                    at org.apache.karaf.features.internal.download.impl.AbstractRetryableDownloadTask.run(AbstractRetryableDownloadTask.java:60) ~[?:?]
                    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
                    at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
                    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
                    at java.lang.Thread.run(Thread.java:829) [?:?]

I couldn’t find a proper guide to fix a 2.x version for the log4j issue… Looks like I’m messingthis up.

Anyone got a clear guide to sort my 2.x for pax/log4j issue?

A lot depends on actual path you put in filesystem and startup properties. Practically filesystem location is not that important cause all metadata is read from actual file. You dont need to create new files, it is sufficient to override existing jar file.

Thanks for the reply and Happy New Year!! :slight_smile:

I’ve messed it up a bit right now. So are you saying it i revert to what it was, then I change just replace the files with 1.11.2 jars with the 1.11.13 jars, but rename them to 1.11.2?

Also, I see that Karaf 4.2.14 has the fix in. Is it possible to overlay the newer version of that over OH 2.x ? If so how?

I’ve downloaded the latest clean 2.5.12 for OH, which still has the bad log4f. So what’s the best way of downloading a clean linux binary install and patching it to be safe?

Sorry for jumping in here again. I’m confused with version numbering. Is there a change log in the marven repository to know which marven version belongs to which apache version?

Only appache log4j version 2.17.1 is bug free at the time of writing this.

What does this mean for OH 2.5.12?
Is Pax logging 1.11.13 the version which is equivalent to 2.17.0 or 2.17.1?

It is a little bit confusing that the numbers are not the same.

I think you can lookup the relation/dependency here:
https://mvnrepository.com/artifact/org.ops4j.pax.logging/pax-logging-log4j2

From what I can see 1.11.13 has 2.17.1 in it.

However, I’m still lost how I change a OH 2.5.12 (which has 1.11.2 in) to upgrade to 1.11.13.

Can someone please explain?

I would have thought a 2.5.13 should be there too, with 1.11.13 in, since this is such a bad issue. 2.5.12, really shouldn’t be used.

In theory third part of version number is irrelevant. In practice it might cause troubles. Most of stuff you see in openHAB or Karaf specify minimum version and as long as you move within x.y range of xy.z version you are on safe side.
Regardless of which exact version of pax logging you use you shall be able to update .z part of version with no major repercussions. My earlier post outlined tips for OH 3.0 but same way shall work for 2.5.x which is on older Apache Karaf version.
My advice (which you already did) - look for pax logging versions your OH 2.5.x uses, take maximum patch (.z) available and override filesystem files. Then do clean start without cache dirs.
Obviously you still need to double check if you get log4j2 >= 2.17.1for pax logging but with little effort you can fix issue without awaiting next 2.5.x release.

I’m not sure the standard install of 2.5.12 is working anymore. I think some online resource is now missing which is needed on first run. So you can’t get a working version from the downloaded binaries. So I can’t even install that to see if the change to that works…

I posted an issue in Installing, for this.

What I’ve done to update to 1.11.13 on OH 2.5.12

Sorry, I’m on windows, but linux should be similar.

Go to that directories and create folders named 1.11.13 on each (similar to the existing ones):
C:\OpenHAB2\runtime\system\org\ops4j\pax\logging\pax-logging-api
C:\OpenHAB2\runtime\system\org\ops4j\pax\logging\pax-logging-log4j2

Download the jar-files from here and put them in the corresponding folter created:
https://repo1.maven.org/maven2/org/ops4j/pax/logging/pax-logging-api/1.11.13/
https://repo1.maven.org/maven2/org/ops4j/pax/logging/pax-logging-log4j2/1.11.13/

You can check the content of the existing folder to place the right jar file in the right folder.

Go to karaf console and enter:
la -l|grep -i pax.logging
6 | Active | 8 | 1.11.2 | mvn:org.ops4j.pax.logging/pax-logging-api/1.11.2
7 | Active | 8 | 1.11.2 | mvn:org.ops4j.pax.logging/pax-logging-log4j2/1.11.2
Important are the leading numbers 6 and 7 (can differ on your side).

Enter that command in karaf (the numbers must correspond to your setup!)
update 6 mvn:org.ops4j.pax.logging/pax-logging-api/1.11.13
update 7 mvn:org.ops4j.pax.logging/pax-logging-log4j2/1.11.13

Now all logging is gone. Restart OH and all is fine, Logging is here again and you can check in karaf if the right .jar files are running

la -l|grep -i pax.logging
should show you version 1.11.13 now.

Most of this post was also written in the post above by splatch.

1 Like

Thank you for that! Much appreciated.

I did this and seemed to work. It now reports;


says 1.11.2 & 1.11.13 on same line?
Is that what it should say? Is it safe now?

Yes, what counts is 4th column (1.11.3) which is taken directly from JAR file. Version you see in last column is URI which is used to retrieve JAR, in practice it points to filesystem location at system/org/ops4j/pax/logging/.... It might be a bit misleading to see two different versions but fastest way to go is simply overriding of existing files. It saves you modification of further files.

thanks! :slight_smile: