GitLab

Upgrades to Apache Tomcat 9.0.54

Removed javascript for jQuery, jQuery Migrate, jQuery Browser and jQuery Validation from OFBiz sources. Instead these are now retrieved during the build as NPM packages using gradle-node-plugin.

Thanks: Aditya Sharma for implementation.

Prevents an useless inevitable warning by commenting out
RequestWrapper::getParameterNames and with it all the unused methods in
RequestWrapper class

Also better comments CacheFilter::doFilter by giving its real and only goal:
<<to prevent a post-auth security issue described in OFBIZ-12332>>

Fixes a typo about filterConfiguration in CacheFilter.java

Forgot the change in build.gradle

We currently have 115 checkstyle errors, most are related to methods using a too
high number of params.

Obviously nobody have currently time to work on this issue.

This commit increases the max ParameterNumber to 26 to hide all current related
errors. This reduces checkstyle errors to 54. It also allows to easier focus on
other errors.
It still possible to works on OFBIZ-12335 by temporary reverting this commit or
replacing max ParameterNumber by the number wanted (was 10, is 7 by default)

In previous commit, in CacheFilter::doFilter, I checked "xmlrpc"  when it was
actually "/control/xmlrpc"

This definitely solves all issues by introducing a CacheFilter and
RequestWrapper classes inspired by several works found on the Net.
Also moves the change introduced before in ContextFilter to CacheFilter.

The basic problem is that you only can use once
ServletRequest::getInputStream or the ServletRequest::getReader
Also not both, even once, ie they can be seen as same from this POV.

The integration tests all pass.

Also replace the checked String "</serializable>" by "</serializable" as, like
reported by Jie Zhu, the former could be bypassed by using "</serializable >"

Thanks: Jie Zhu for report

Temporarily comments out XMLRPC tests.

I'll work on a definitive solution ASAP

As reported by Jie Zhu:
<<The latest version of the OFBiz framework (17.12.08) is affected by an
XMLRPC Remote Code Execution Vulnerability.
This vulnerability is caused by incomplete patch repair of cve-2020-9496.>>

Actually this is not an OFBiz bug (so not related to CVE-2020-9496)
but an old XMLRPC bug (Archiva was(/is?)) also affected:
https://nvd.nist.gov/vuln/detail/CVE-2016-5003

Unfortunately XMLRPC is no longer maintained, so it's OFBiz responsibility to
fix this bug.

As the code that secures serialisation in OFBiz is not reached by this bug, the
solution is to secure it at the ContextFilter class level (ie before it reaches
secured serialisation in OFBiz source).

Thanks: Jie Zhu for report and help.

Fixed: Edit record in product promotion, "Promotion Last Modified Date" is invalid, but don't notice to user (OFBIZ-12046)

As Lalit reported it's not really a bug. But an annoying service transformation
of updateProductPromo to entity-auto (I checked using R15.12). Note that
createProductPromo is not affected

I'm not sure other update services using entity-auto are not affected

Thanks: Do Nhu Vy for report, Lalit Dashora  for initial analysis

Weirdly (I guess I did not save) this syntax did not work 1st time I tried it

One less (from 116 to 115), not sure which one :)

Some parties have no PartyAccountingPreferences

Thanks: Yashwant Dhakad for report (see "Steps to reproduce" in Jira)

Jira title: OutOfMemory and stucked JobPoller issue

On ecommerce website with strong frequentation and shopping cart saving, when the service autoDeleteAutoSaveShoppingList run, it failed due to the data quantity to delete and sature the server ressources.

The solution to escape this case is :
 * filter shopping list to delete directly by the data search
 * split quantity by transaction

Thanks: Giulio Speri

The class PartyHelper is massively used to resolve the name of a party without know is type.

     <field name="organizationPartyId" title="${uiLabelMap.ProductOrganization}">
         <display description="${groovy: org.apache.ofbiz.party.party.PartyHelper.getPartyName(delegator, organizationPartyId, true);} [${organizationPartyId}]"/>
     </field>

A party name have few change over time, or the PartyHelper.getPartyName function call the database at each call.

On huge screen, we distinctly clearly the database latency.

So for a cold data, no reason to didn't use the cache.

Correction for content and accounting AR & AP

Fixed: ShoppingCart object does not recognize two products with different configurations (OFBIZ-12303)

By adding two equal products with different configurations in eCommerce cart it results in qty aggregation instead of adding 2 separate cart lines.

When we added a new configuration in cart, we analyze all configuration options already present to know if this configuration case is alredy load and merge the quantity added.

Thanks: Alexander Tzvetanov for raise this issue

No functional change, review the code to down less it :
 * Use groovy syntax
 * Use DSL when is not use as it should
 * Add missing variable type, for most ide analyser potential

If you set explodeItems to true on your productStore, tax and adjustment wasn't correctly calculate when you ordered some finish goods.

Thanks: Suraj Khurana and Ankush Upadhyay

It's possible that create a Telecum number without contactNumber that appears as an inconsistency.

Thanks: Rashi Dhagat for this issue

When the webshop (ecommerce component) has been set up for retail (B2C), with

    the tax setting 'Show prices with VAT tax included' set to 'Y' in store 9000
    the appropriat Tax Auth Geo and Tax Auth set in store 9000
    the flag 'Include Tax In Price' set to 'Y' for the Tax Auth
    the price for a virtual product is shown including VAT.

However, for a selected variant the price shown is without VAT

Thanks: Pierre Smits to raise the issue and Priya Sharma for solved it

Previous approach to rendering FTL macros was to encode all macro
arguments as a single string and pass to the FTL processor.

New approach is to use FTL's default wrapper to handle passing Java
types to the FTL processor and referencing them from macro calls.

Thanks: Xin Wang for introducing FTL with_args approach and providing
POC.

At https://markmail.org/message/inoihmcon37wq5zz I suggested:

If I'm not wrong, we no longer use Docbook anywhere. So I suggest to remove it
from:

1. cmssite component, where it was used by the online help, now assured by
    Asciidoc. BTW, for whom it could concern, I noticed that anchors jumping
    in adoc files does not work in online help
2. docbook.dtd in content component only used by the cmssite component
3. docbook.css in themes, I'm not sure that still useful there

Replace java 9 `List.of` that is not yet compatible with current CI java
configuration (java 8)

Improve readability using stream api, while adding unit test for
refactoring validation.
Fix deniedWebShellTokens property where some blank space where removed
and '// python' comment was inlined.

Improved: Adds the HTML <input> accept Attribute in form widgets and Freemaker templates (OFBIZ-12049)

Adds HTML accept where it fits, let the more general cases as they are

Extracts the list of words used in SecuredUpload::isValidText in a
deniedWebShellTokens property in security.properties

(OFBIZ-11446)

Also change the reference in DataResourcePermissionServices.xml for checkOwnership

This much increases the current security by creating SecuredUpload::isValidText
and call it from ProgramExport.groovy

This said I agree with thiscodecc that a better solution would be to use a
Groovy sandbox, and not only in ProgramExport.groovy.

I had a quick glance at a such solution. Unfortunately as Cédric Champeau reports
at https://melix.github.io/blog/2015/03/sandboxing.html and unlike thiscodecc
suggest there are no "mature solutions on the market".

Nevertheless I'll have a deeper look at an OFBiz specific Groovy sandbox
solution.

Somehow related: I'll also soon extract the list of words used in
SecuredUpload::isValidText in a deniedWebShellWords property in security.properties

Thanks: thiscodecc for report

Correct previous commit with remove empty file InvoiceEvents.xml, reoriented the function createInvoiceItemPayrol from InvoiceServices.groovy to new file InvoiceEvents.groovy

Thanks to Nitish Mishra for started this issue

No Functional change

When you call a groovy script from an event controller, some information are present note on the same place on the binding context.

Example if you call a groovy script as service you found the userLogin with parameters.userLogin or when you call it as event, the userLogin is on the binding context root.

The problem appear with the DSL method 'run service' who search the missing value need by a service (userLogin, locale and timezone) on the map parameters on the binding context, so failed to populate correctly the information for an event.

Call from event
    Map serviceResult = run service: 'createInvoice', with: [partyId: partyId, invoiceDate: nowTimestamp]

Failed due to security issue : missing userLogin on the service context.

Fixes a number of issues I spotted while working on OFBIZ-12305 in relation with
OFBIZ-12055

The last change I made for OFBIZ-12305 was incomplete, the files could not be
checked by SecuredUpload because they did not exist! This concerns
ImageManagementServices, DataServices, FrameImage and ProductServices classes.
I used Files::createTempFile to fix that.

Also fixes a bug in SecuredUpload where I reversed the check on
fileToCheck.length in Windows case. I also added a comment, Windows 10 now
allows more length (need an OS parameter change though)

Finally, creates public SecuredUpload::isValidText to be used in OFBIZ-12305

reload the last commit faa59b32 reverted with improve the groovy syntax and remove the empty fiels PaymentServices.xml

Thanks: Sourabh Punyani

Improved: Convert checkAndCreateBatchForValidPayments service from mini-lang to groovy DSL (OFBIZ-11497)

Thanks to sourabh jain I started a solution with your patch

Thanks to sourabh jain for the patch

Thanks to Rohit Koushal to submit the patch

Improved: Convert createMatchingPaymentApplication service from mini-lang to groovy DSL (OFBIZ-11500)

Thanks to Rohit Koushal to submit the patch