slow calculate_totals() – tips?

[Pablo]

My store is heavily customized.

It only shows cart products and their prices in the actual /cart/ and /checkout/ pages.

When any page is loaded, Woo inits the WC_Cart object and runs calculate_totals().

This function is performing poorly because of this plugin.

Here’s a sample trace overview:

And a drill down of that 2.190ms do_action:

In it we can see how 40% of the request processing is for WJECF_AutoCoupon::get_valid_auto_coupons

I’m trying to come up with a solution to that.

I think that aborting the call to calculate_totals() on any page other than /cart/ and /checkout/ is the most direct way, but i’m not sure if that’s possible. While i understand that that’s more of a Woo question, with your expertise can you tell me if that’s possible / how to go about doing that?

I think that the next best approach is to limit that idea to just the plugin. calculate_totals() still runs, but the extended coupon features are not checked for.

Again, thoughts on feasability / implementation ?

Thanks in advanced for the attention!

[Soft79]

How many auto coupons do you have configured?

Can you paste your WC System Status Report?

[Pablo]

Funny enough, i have exactly 4.

### WordPress Environment ###

Home URL: […]
Site URL: […]
WC Version: 3.3.5
Log Directory Writable: ✔
WP Version: 4.9.5
WP Multisite: –
WP Memory Limit: 800 MB
WP Debug Mode: ✔
WP Cron: –
Language: en_US

### Server Environment ###

Server Info: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips
PHP Version: 7.1.16
PHP Post Max Size: 8 MB
PHP Time Limit: 600
PHP Max Input Vars: 1000
cURL Version: 7.29.0
NSS/3.28.4

SUHOSIN Installed: –
Max Upload Size: 8 MB
Default Timezone is UTC: ✔
fsockopen/cURL: ✔
SoapClient: ❌ Your server does not have the SoapClient class enabled – some gateway plugins which use SOAP may not work as expected.
DOMDocument: ✔
GZip: ✔
Multibyte String: ✔
Remote Post: ✔
Remote Get: ✔

### Database ###

WC Database Version: 3.3.5
WC Database Prefix: […]_
Total Database Size: 19389.14MB
Database Data Size: 18623.14MB
Database Index Size: 766.00MB
[…]_woocommerce_sessions: Data: 10.55MB + Index: 0.23MB
[…]_woocommerce_api_keys: Data: 0.02MB + Index: 0.03MB
[…]_woocommerce_attribute_taxonomies: Data: 0.02MB + Index: 0.02MB
[…]_woocommerce_downloadable_product_permissions: Data: 2.52MB + Index: 3.33MB
[…]_woocommerce_order_items: Data: 22.55MB + Index: 9.52MB
[…]_woocommerce_order_itemmeta: Data: 17649.83MB + Index: 120.30MB
[…]_woocommerce_tax_rates: Data: 4.52MB + Index: 6.06MB
[…]_woocommerce_tax_rate_locations: Data: 2.52MB + Index: 5.55MB
[…]_woocommerce_shipping_zones: Data: 0.02MB + Index: 0.00MB
[…]_woocommerce_shipping_zone_locations: Data: 0.02MB + Index: 0.05MB
[…]_woocommerce_shipping_zone_methods: Data: 0.02MB + Index: 0.00MB
[…]_woocommerce_payment_tokens: Data: 0.02MB + Index: 0.02MB
[…]_woocommerce_payment_tokenmeta: Data: 0.02MB + Index: 0.03MB
[…]_woocommerce_log: Data: 0.02MB + Index: 0.02MB
[…]_ac_abandoned_cart_history_lite: Data: 25.55MB + Index: 0.00MB
[…]_ac_email_templates_lite: Data: 0.02MB + Index: 0.00MB
[…]_ac_guest_abandoned_cart_history_lite: Data: 1.52MB + Index: 0.00MB
[…]_ac_sent_history_lite: Data: 0.13MB + Index: 0.00MB
[…]_commentmeta: Data: 0.02MB + Index: 0.03MB
[…]_comments: Data: 32.56MB + Index: 21.09MB
[…]_links: Data: 0.02MB + Index: 0.02MB
[…]_options: Data: 15.52MB + Index: 0.16MB
[…]_pantheon_sessions: Data: 6.31MB + Index: 0.98MB
[…]_postmeta: Data: 397.98MB + Index: 404.61MB
[…]_posts: Data: 18.55MB + Index: 12.06MB
[…]_termmeta: Data: 0.02MB + Index: 0.03MB
[…]_terms: Data: 0.02MB + Index: 0.03MB
[…]_term_relationships: Data: 0.17MB + Index: 0.11MB
[…]_term_taxonomy: Data: 0.02MB + Index: 0.03MB
[…]_usermeta: Data: 344.97MB + Index: 149.41MB
[…]_users: Data: 7.52MB + Index: 7.55MB
[…]_wc_download_log: Data: 1.52MB + Index: 0.52MB
[…]_wc_webhooks: Data: 0.02MB + Index: 0.02MB
[…]_yoast_seo_links: Data: 0.02MB + Index: 0.02MB
[…]_yoast_seo_meta: Data: 0.02MB + Index: 0.00MB

### Post Type Counts ###

acf-field: 65
acf-field-group: 8
attachment: 22
home_slider: 4
page: 14
post: 15
product: 31
shop_coupon: 7411
shop_order: 45150
shop_order_refund: 394
subproducts: 48
wccpf: 5
wpcf7_contact_form: 2

### Security ###

Secure connection (HTTPS): ❌Your store is not using HTTPS. Learn more about HTTPS and SSL Certificates.
Hide errors from visitors: ✔

### Active Plugins (18) ###

Native PHP Sessions for WordPress: by Pantheon – 0.6.6
Admin Columns Pro: by Admin Columns – 4.2.4
Advanced Custom Fields PRO: by Elliot Condon – 5.6.10
Disable Comments: by Samir Shah – 1.7.1
JWT Authentication for WP-API: by Enrique Chavez – 1.2.4
Post Types Order: by Nsp Code – 1.9.3.6
Product Sales Report for WooCommerce: by Potent Plugins – 1.4.8 – Not tested with the active version of WooCommerce
User Role Editor: by Vladimir Garagulya – 4.40.3
WC Fields Factory: by Saravana Kumar K – 2.0.7 – Not tested with the active version of WooCommerce
WC Search Orders By Product: by Akshaya Swaroop – 1.1 – Not tested with the active version of WooCommerce
WooCommerce Bulk Email: by Jaydeep Rami – 1.0.0 – Not tested with the active version of WooCommerce
Advanced Order Export For WooCommerce: by AlgolPlus – 1.5.3
Abandoned Cart Lite for WooCommerce: by Tyche Softwares – 4.8 – Not tested with the active version of WooCommerce
WooCommerce Extra Fee Option: by Terry Tsang – 1.0.7 – Not tested with the active version of WooCommerce
WooCommerce Authorize.Net AIM Gateway: by SkyVerge – 3.13.0 – 3.14.1 is available
WooCommerce: by Automattic – 3.3.5
WP Mail SMTP: by WPForms – 1.2.5
WP Migrate DB: by Delicious Brains – 1.0.2

### Settings ###

API Enabled: ✔
Force SSL: ✔
Currency: USD ($)
Currency Position: left
Thousand Separator: ,
Decimal Separator: .
Number of Decimals: 2
Taxonomies: Product Types: external (external)
grouped (grouped)
simple (simple)
variable (variable)

Taxonomies: Product Visibility: exclude-from-catalog (exclude-from-catalog)
exclude-from-search (exclude-from-search)
featured (featured)
outofstock (outofstock)
rated-1 (rated-1)
rated-2 (rated-2)
rated-3 (rated-3)
rated-4 (rated-4)
rated-5 (rated-5)

### WC Pages ###

Shop base: #442 – /
Cart: #8366 – /store-home/cart/
Checkout: #8367 – /store-home/checkout/
My account: #8368 – /my-account/
Terms and conditions: ❌ Page not set

### Theme ###

Name: […]
Version: 1.0
Author URL: […]
Child Theme: ✔
Parent Theme Name: Salient
Parent Theme Version: 7.0.5
Parent Theme Author URL: http://themenectar.com
WooCommerce Support: ✔

### Templates ###

Archive Template: Your theme has a woocommerce.php file
you will not be able to override the woocommerce/archive-product.php custom template since woocommerce.php has priority over archive-product.php. This is intended to prevent display issues.

Overrides: salient/woocommerce/cart/cart-shipping.php version 2.5.0 is out of date. The core version is 3.2.0
salient-child/woocommerce/cart/cart-totals.php
salient-child/woocommerce/cart/cart.php version 2.3.8 is out of date. The core version is 3.3.0
salient-child/woocommerce/cart/mini-cart.php version 2.5.0 is out of date. The core version is 3.3.0
salient/woocommerce/cart/shipping-calculator.php version 2.0.8 is out of date. The core version is 3.2.0
salient-child/woocommerce/checkout/form-billing.php version 2.1.2 is out of date. The core version is 3.0.9
salient-child/woocommerce/checkout/form-checkout.php
salient-child/woocommerce/checkout/form-coupon.php version 2.2 is out of date. The core version is 3.3.0
salient-child/woocommerce/checkout/form-shipping.php version 2.2.0 is out of date. The core version is 3.0.9
salient-child/woocommerce/checkout/payment-method.php
salient-child/woocommerce/checkout/review-order.php version 2.3.0 is out of date. The core version is 3.3.0
salient-child/woocommerce/checkout/thankyou.php version 2.2.0 is out of date. The core version is 3.2.0
salient/woocommerce/content-product.php version 2.6.1 is out of date. The core version is 3.0.0
salient-child/woocommerce/content-single-product.php version 1.6.4 is out of date. The core version is 3.0.0
salient-child/woocommerce/emails/email-order-details.php version 3.0.0 is out of date. The core version is 3.3.1
salient-child/woocommerce/emails/email-order-items.php version 3.0.0 is out of date. The core version is 3.2.0
salient-child/woocommerce/global/quantity-input.php version 2.5.0 is out of date. The core version is 3.3.0
salient/woocommerce/loop/add-to-cart.php version 2.5.0 is out of date. The core version is 3.3.0
salient-child/woocommerce/myaccount/downloads.php version 3.0.0 is out of date. The core version is 3.2.0
salient/woocommerce/myaccount/form-login.php version 2.6.0 is out of date. The core version is 3.3.0
salient-child/woocommerce/myaccount/form-lost-password.php version 3.0.0 is out of date. The core version is 3.3.0
salient-child/woocommerce/myaccount/form-reset-password.php version 3.0.0 is out of date. The core version is 3.3.0
salient-child/woocommerce/myaccount/lost-password-confirmation.php
salient-child/woocommerce/myaccount/my-account.php
salient/woocommerce/single-product/product-image.php version 2.6.3 is out of date. The core version is 3.3.2
salient/woocommerce/single-product/tabs/description.php
salient/woocommerce/single-product/tabs/tabs.php

Outdated Templates: ❌Learn how to update

### Authorize.Net AIM ###

Environment: Production
Debug Mode: Save to Log

### Authorize.Net AIM eCheck ###

Environment: Production
Debug Mode: Save to Log
`

[Soft79]

Extended Coupon Features is not in the Status Report. What version is it? Is it PRO or FREE?

[Pablo]

I’m very sorry! Forgot it was disabled during my back-and-forth tests. It’s PRO, version 2.5.4.

[Soft79]

– First update to newest version. Current is 2.6.2

– Can you enable debug mode of our plugin; run a couple of test and then send us the log? (WooCommerce > Status > Log > WooCommerce-Extended-Coupon-Features-……..

– What profiling tool are you using to generate that drill down?

– Your child theme is VERY outdated. Let your theme author fix this.

[Pablo]

* I’m using new relic and xhprof for profiling
* I’m the child theme author. I keep tabs on woo theme updates but have to admit that i’ve been lazy to change their version number. I’ve also recently refactored the child theme to be a full one, getting rid of the salient dependency (and overhead)
* I had not noticed how ECF was not in its latest version

Updating ECF will take some time as we’ve built some custom functionality around it. I’ll be doing that and returning to this topic when there’s an update.

For now i thank you so much for the agile responses and hope that, should my issues persist after the update, we can continue the conversation!

[Soft79]

Another question; can the drill down be further expanded in the methods get_all_auto_coupons and coupon_can_be_applied?

How did you customize ECF? Did you alter the plugin itself? That’s not recommended.

[Pablo]

I’ve made slight customizations to it to attend to this store’s needs, none have affected performance (i’ve measured them). But to rule out the intrincacies of my custom installation, i went ahead and attempted to isolate the problem (Something that, in retrospect, i see i should’ve done from the beggining). Blank woo with storefront install + latest ECF + a few (6) coupons set to auto. While the amount of miliseconds decreased (to around 400), the trend persisted: update_matched_autocoupons() continued to take around 40% of the server response time.

40% is something that i think we can live with on cart and checkout pages, but not product listing and detail pages.

The biggest question i’m asking myself is: does update_matched_autocoupons have to be hooked to woocommerce_after_calculate_totals? I see that the plugin was coded defensively with the use of add_action_once, but, i can’t help but wonder if that could be further improved, ie: only reevaluate coupons whenever something about the cart does change, and only then. Perhaps adding a middleman? The middleman could be hooked to woocommerce_after_calculate_totals in order to preserve the desired execution order, but it would check wether coupons do need reevaluating before calling update_matched_autocoupons(). Or something along this line.

[Soft79]

The 100s of milliseconds is just not right. Measuring on my system the call is never more than 4 ms. That’s why I ask you for a further drilldown of get_all_auto_coupons() and coupon_can_be_applied().

Those two methods don’t do a lot of work, but they can create a new instance of WC_Coupon. So, possibly another hook (from a different plugin) is fired which slows things down. Is your database on a local server or remote? Can the database be the culprit?

This is part of the measurement on my system:

Elapsed in get_all_auto_coupons: 0.382261 - 0.380914 = 0.001347 sec
Elapsed in get_all_auto_coupons: 0.384839 - 0.382290 = 0.002549 sec
Elapsed in get_all_auto_coupons: 0.385457 - 0.385452 = 0.000005 sec
Elapsed in get_all_auto_coupons: 0.412427 - 0.412420 = 0.000007 sec
Elapsed in get_all_auto_coupons: 0.418027 - 0.412456 = 0.005571 sec
Elapsed in get_all_auto_coupons: 0.418671 - 0.418666 = 0.000005 sec
Elapsed in get_all_auto_coupons: 0.465769 - 0.465758 = 0.000011 sec
Elapsed in get_all_auto_coupons: 0.470228 - 0.465832 = 0.004396 sec
Elapsed in get_all_auto_coupons: 0.471185 - 0.471180 = 0.000005 sec

Changing the hooks used by the plugin is a risky operation with 10.000’s of installations; I do have plans on changing this but it will probably be in a 3.0 release.

[Soft79]

Did you investigate this further? Could you find anything?

I’m working on a new concept for the auto-coupon application. If can can try it out to see if it improves things, it would be great. Please paste the contents from https://pastebin.com/mCdh3Euw in includes/WJECF_AutoCoupon.php and let me know the result.

[Soft79]

Development version 3.0.0-dev.2 is available on your account page. Might have a significant performance improvement in your case.

Please let me know.

[Pablo]

Man this is tough. You’re right in saying that 100ms is just not right, and your numbers made me rethink all assumptions. I couldn’t get a further drilldown of that particular trace i initially showed, but you were right in thinking there’s other stuff happening triggered by ECF. I use the abandoned carts plugin, and it does its thing (reads and writes from its table) upon every cart change. It considers ECF’s resetting of autocoupons as cart changes. I have a discussion with that plugin’s writer similar to ours, in which he’s looking into using less frequently fired hooks. In the meanwhile i’ve come up with a hack for that plugin, to throttle its execution: that helped a bit.

There’s also code of mine that was running too often. I’ve applied a throttling technique to it similar to your add_action_once wrapper, and that helped a bit.

There’s code of mine hooked into woocommerce_product_get_price that runs all the time, but it is unavoiadable. It’s simple in its nature, but essential for our business rules: it checks for a session variable and then overwrites the price with a different one that’s saved in product meta’s, should the condition be true.

I’ve experimented with object cache via redis, that also helped a bit.

Oh! One of the things that does give ECF a hard time is my somewhat modified version of the popular woocommerce_maybe_add_multiple_products_to_cart hack. We don’t add products each at a time, we do it in bulk when the user presses a button. This gives a snappier UX, but kills the backend because once the bulk request is submitted we have to go through woo’s native add_to_cart for each of them, which triggers ECF reprocessing for each product.

As for the database, it’s a cloud db instance – different/dedicated server but on the same private network. It’s on a bulky rig with 16gb of ram.

You know – i also appreciate you calling out the instantiation of WC_Coupon. I think there’s a problem there.
We have 4 autocoupons, but they’re frequently applied and we have around 1k daily orders. This means that that darned “_used_by” prop is HUGE for them, and getting bigger by the minute. I hate that prop with a passion lol.

I apologize for going quiet. I’ve looked at so many things. Not to mention infrastructure stuff like changing from apache to nginx, tweaking fpm and opcache.

It’s been hard to sit down and write back. Every time i started to i was like “hmmm but i haven’t checked x yet, let me do that first”.

And i appreciate your patch! So much. I’ll definitely take it for a spin. It might just take me a little while to write back, but the quest continues.

[Soft79]

Any news on testing 3.0.0-dev?

[Leandro Alves]

hi, nice speed tips here:

https://fixmysite.com/slow-woocommerce-website-fix/

[Author]

Posts