Can You Test Emptiness of Laravel Collection using empty()?

In short, Yes and No. Not the way we commonly do a variable, but in laravel way yes. It’s a common mistake done by almost all the laravel developer once in a lifetime until the bug appears (Well, you are not counted, if you are exceptional :P). So, let’s explore.

Let’s look at how laravel collection is constructed. Go to your laravel tinker console and try this:

php artisan tinker
Psy Shell v0.9.12 (PHP 7.2.31 — cli) by Justin Hileman
>>> $collection = collect([])
=> Illuminate\Support\Collection {#3164
     all: [],
   }

You see, when I create an empty collection, laravel still puts an underlying array called ‘all’. This is the manipulator array and contains all the content inside. This array is accessible through the collection all method:

>>> $collection->all()
=> []

You see, the return is an empty array. But when it’s just the collection, it’s not really empty, it has an underlying content holder.

So, how can we test emptiness of the collection? Well, there are 3 ways.

I) Laravel gives a way to return the number of element in the collection with a collection method calls count(). You can test it against 0 to see if the collection is empty or not

>>> $collection->count()
=> 0

II) You may use the regular php count() method to return that it doesn’t contain any leaf element and test it against 0:

>>> count($collection)
=> 0

III) If you are a big fan of ’empty’ and still would like to follow, then you can grab the content of the collection using all method and test it against empty as following:

>>> empty($collection->all())
=> true

So, yeah, now you know all the ways 🙂

How To: Restore Zimbra Quarantined Email by Clam AKA Heuristics.Encrypted.PDF Release Point

Zimbra Mail Server automatically quarantines emails that get hit by the Antivirus scan using Clam when the mail is received. While putting the email on the recipient inbox, what it does, instead of giving the original email with the attachment, it sends a virus detected email with the following kind of error message:

Virus (Heuristics.Encrypted.PDF) in mail to YOU

Virus Alert
Our content checker found
virus: Heuristics.Encrypted.PDF

by Zimbra

It actually means, the original mail is now quarantined. Zimbra maintains a virus quarantine email account that is not normally available in the ‘Manage Account’ list of Zimbra Admin panel. You can find it if you search with ‘virus’ in the ‘Search’ box of the admin panel. What zimbra does in quarantine situation, is that, it pushes the mail to the quarantine email instead of original recipient.

Now, to get back the mail delivered to the original recipient, we need to first get the quarantine email account, get the message id, and then we need to inject the mail into the LMTP pipe that bypasses any scanning. Here are the steps on how to do this:

# First get to the zimbra user
$ su - zimbra

# Get the email account that is used to store virus detected mails
$ zmprov gcf zimbraAmavisQuarantineAccount
zimbraAmavisQuarantineAccount: virus-quarantine.mrhmcbg4kz@yourhostname.com

# virus-quarantine.mrhmcbg4kz@yourhostname.com this should be our quarantine email account, now we need to get the quarantine account's mailbox id
$ zmprov gmi virus-quarantine.mrhmcbg4kz@yourhostname.com
mailboxId: 73
quotaUsed: 644183

# Mailbox id here for the quarantine account is 73. Now go to the message storage of this id using the following command: cd /opt/zimbra/store/0/<mailboxId>/msg/0
$ cd /opt/zimbra/store/0/73/msg/0

# list the messages
$ ls *

These are your quarantined emails. Now for example the complainer is ‘sam@yourhostname.com’. To search for the emails designated for this email account, you may use the following:

$ grep -l sam@yourhostname.com *
281-1216.msg
300-1400.msg
301-1476.msg

This should return you all the emails that got quarantined for the above user.

Now the question is, how can we get these emails delivered to the designated user bypassing the antivirus/antispam tools. To do this, you need to inject the mail into LMTP pipe. You may do this using ‘zmlmtpinject’ command as following:

$ zmlmtpinject -r sam@yourhostname.com -s admin@yourhostname.com 281-1216.msg

Remember, to change same@yourhostname.com to the original recipient. admin@yourhostname.com would be the newly rewritten sender for this mail delivery and ‘281-1216.msg’ is the file name of the original email that you found out from the grep command. You can do lmtp injections for one email mail with each command. So, you would require to do this for each emails.

How to Do Full Page Caching in Laravel / How to Cache Views in Laravel

Most of the developers use Laravel Cache for database query result caching. Although, this is efficient, but the ultimate caching performance enhancement is achieved through FPC or Full Page Caching for web apps. Laravel doesn’t give any hint, neither describe how to do this in their documentation, which is why the article.

What is Full Page Caching?

Technically a full page caching means, to cache the html response from an app. In FPC, it is generally accepted to use the route/view as the cache key concatenating or mixing with the VERB in request header.

When a user requests for a route, we usually pull a controller behind the route to process and prepare several data before sending them to views for response. But what if the data hasn’t changed since the last request? That technically means the response hasn’t changed, right? This essentially says, you can cache the full response and skip the whole controller processing, even pulling the view, instead, only put the Cached data in the response. Theoretically, this is the best form of caching mechanism for ‘Web Based’ solutions like Ecommerce, Newspapers, Blogs etc. This technique is known as FPC or Full Page Caching.

Laravel Cache

Laravel is best known for it’s documentation. Although, the Laravel Cache documentation, only follows how to cache the database queries, not the views. To understand how to do FPC using Laravel, let’s first look at how our views are usually formed.

class NewsController extends Controller {
    public function index() {
        $news = News::all();
        return view('news.index')->with('news', $news);
    }
}

Here the view() helper method, returns a Laravel View instance. It doesn’t return the html or renders one. So who does it? Laravel does it for you under the hood, and pass it to Response class. Now to cache the views, you have to return the html and save it to cache. There are basically two ways of doing it.

The easiest way is to use a function called ‘render()’ that is available to View class which returns the html of the created View instance. Here is how you may convert the above controller method to return from cache:

class NewsController extends Controller {
    public function index() {
        if ( Cache::has('news_index') ) {
            return Cache::get('news_index');
        } else {
            $news = News::all();
            $cachedData = view('news.index')->with('news', $news)->render();
            Cache::put('news_index', $cachedData);                                         
            return $cachedData;           
        }  
    }
}

This should be it, simple, ha!

Here is more! I looked at the laravel documentation a bit more, and I could find there is another way you can do the above. This is using the Response class. view method returns a Views instance, while Response instance is able to return rendered html based on view. Here is how to do this:

Response::view('news.index')->with('news', $news);

This also means our idea that Laravel does the rendering under the hood is a bit wrong, it basically shoots the views instance to a response instance (which it has to) and returns it, that put the rendered html in the final response. We can now cache the above output and serve for future requests without entering the controller’s processing!

Dirty Odoo Hacks: How to Know If a Scheduled Action is Running

This blog post goes to my ‘Dirty Hack’ series, where I try to open the gross hacking attempts I practically use in several of my projects.

Odoo is my favorite piece of framework. Even though it has evolved as OpenERP, but the extend ability that Odoo gives, probably can take it anywhere, any other framework unable to, without massive change to their base architecture. This goes to the pros of Odoo, while the biggest cons of Odoo, is that it is not owned/developed by a major ‘English’ speaking company/people. If you go through the Odoo documentation, you can clearly understand the difference I am talking about. One key reason, Odoo is not often seen everywhere, probably because of poor documentation. Most of the things are not explained in details, as Odoo is a large piece of software, makes it difficult for developers to dig it down and extend.

Odoo has it’s own scheduling system, which they call ‘Scheduled Actions’. Odoo manages a model ‘ir.cron’ to manage the Scheduled Actions. If you look at the model details from (on debug mode)
Odoo >> Settings >> Technical >> Database Structure >> Model
you will see, there is no field it uses to detect if the scheduled action is running or stopped. It has a ‘state’ property though, which does not mean the status like many other odoo models. Here is a output of state property from Odoo shell:

>>> cron_id = self.env['ir.cron'].browse(18)
>>> cron_id.state

'code'

That says, it is telling you what kind of operation it is going to execute, here it is saying, this scheduled action is executing a ‘Python Code’ here.

There are times, when you might require to determine whether the scheduled action is running or not. In my case, it was to track down the some syncing issues, like to find out whether the syncing is doing it’s job properly through some custom methods. As Odoo doesn’t provide a way to do this, I had to find a dirty way to do it. Here is how I did it.

Odoo runs in Postgres SQL, which is basically an open source version of Oracle like Object Oriented Database system. Like many other OODS, Postgres also provide an operation called the following:

FOR UPDATE NOWAIT

From the oracle documentation, it means:

Oracle provides the FOR UPDATE NOWAIT clause in SQL syntax to allow the developer to lock a set of Oracle rows for the duration of a transaction.

When the scheduled action runs, it locks the specific row of the ir_cron table, of course it would because it has to update that specific row with nextcall and other field data. So if a cron is running, and you try to put a lock with another process, will basically fail, which means the cron is running, otherwise the opposite is true. Viola, that should work for us, isn’t it? Simple! Then again, you have to remember, the lock will put in place for the amount of time you specify, or you have to throw an early rollback (why not a commit? to be in safety to avoid any unwanted data being committed to the database during that session). Here is the simple method to do the whole process:

def _check_if_cron_running(self):
cron_id = self.env['ir.cron'].browse(18)
""" assuming the cron id is 18, you can do any kind of search, like searching by name of the scheduled action etc """
if cron_id:
try:
self._cr.execute("""SELECT id FROM "%s" WHERE id IN %%s FOR UPDATE NOWAIT""" % cron_id._table, [tuple(cron_id.ids)], log_exceptions=False)
""" self._cr.execute is used to execute a direct sql command, each ir.cron model data will have 'ids' to tell you which ids are selected, and _table to tell you what is the table name for the model, here it should return ir_cron """
self._cr.rollback() # we need to rollback and give the database cursor back to the other process ASAP
_logger.info("log and operate whatever here, this section is reached when the scheduled action is stopped or not running")
except psycopg2.OperationalError:
self._cr.rollback() # we need to rollback the errors and give control to the other process
_logger.info("log and operate whatever here, this section is reached when the scheduled action is running")

Remember, this is dirty, you are hitting direct SQL, on a near around 500 model based framework. So, please take care of your things before placing this in production. Rest assured, keep using Odoo for future! A brilliant piece of art is Odoo!

How to Empty a Model In Odoo / Mass Delete in Odoo / Mass Unlink

In Odoo, we use unlink() ORM method to delete a record. But if you are trying to empty a model or wants to delete multiple records, the best way to do it, is to use two steps.

First, search the records:

record_set = self.env['your.model'].search([])

Second, unlink them all at once instead of looping through them:

record_set.unlink()

If you have a list of ids, search them using the list:

ids = [1, 2, 3, 10, 11, 12]
record_set = self.env['your.model'].search([('id', 'in', ids)])

and unlink:

record_set.unlink()

Remember, there is no need to loop through this iterable object, odoo unlink does it for you.

TIPS: If you want to get all the ids in a model directly in a list, you can use the following:

record_list = self.env['your.model'].search([]).ids

How to Hide a Column Dynamically in a Parent Model in Tree View on Odoo

We all know how to hide a column in a model based on parent value matching in Odoo installation. I have already written a post on this here:

How to hide a column ‘dynamically’ in Tree View on Odoo

Above case works when you are trying a hide a column in a sale order line, or move line or invoice line. But what if, you want to hide a column in stock picking delivery tree view or the receipt view? The above method, will not work, because column_invisible only works when the parent is referenced. But in a tree view list like ‘Deliver Orders’ under Inventory Overview, if you want to hide a column based on condition, how do we do that?

The answer is, we use data from context and set the attribute ‘invisible’ to True based on the context value. We can either do that by using any existing context value or we can add a context value using a computed field property and match it in the xml.

I will discuss on how can we use the existing context key:value pair in hiding a column on stock.picking model.

First, let’s imagine, we have a character field on stock.picking called ‘woocommerce_id’.

class stock_picking(models.Model):
_inhert = 'stock.picking'

woocommerce_id = fields.Char(string="Woocommerce ID")

Now the xml for Inventory overview stock.picking trees would be inheriting stock.vpicktree

<record id="inherit_delivery_picking" model="ir.ui.view">
<field name="name">stock.picking.inherit.delivery.picking</field>
<field name="model">stock.picking</field>
<field name="inherit_id" ref="stock.vpicktree"/>
<field name="arch" type="xml">
<xpath expr="//field[@name='location_dest_id']" position="after">
<field name="woocommerce_id"/>
</xpath>
</field>
</record>

Now, the above will show woocommerce_id in all vpicktree view, like ‘Delivery Orders’, ‘Receipts’, ‘Internal Transfers’. But What if, you want to show this field to only Delivery Orders?

To do that, we need to know the value for ‘default_picking_type_id’. This value is automatically set on stock.picking model view in the context by Odoo (Hints: You may view all the keys available to context in a model, by creating a computed field, that writes self.env.context.keys() to a file or show it in the tree view to find the keys, and self.env.context.get(‘key_name’) to find the value for it).

If you check the different values set by that key for different pages, you can see, it is set to 1 for delivery orders, 3 for receipts and 5 for internal transfers. Now if you want to show the woocommerce_id field to only delivery receipts, we set the attribute invisible for that field when the context key value is not 1 like following:

<field name="woocommerce_id" invisible="context.get('default_picking_type_id') != 1"/>

Save it, upgrade the module, and see the magic! cheers!

How to Update Context in Odoo

You may want to pass some data to a specific page in Odoo, and change the fields based on those data. In those cases, you want to use Context. Context is a data structure passed to the request object that contains hidden values, and can be used to change results of the page.

Context is a frozendict data type in Odoo. That’s why you can change it like you do in a dict data type, for example:

dict_object.update({
'test': 'test_value',
})

As the Context is frozendict, it won’t take such changes. Odoo provides a way to change values, it’s called ‘with_context’. The syntax is as following:

self = self.with_context({
'test': 'test_value',
})

This one should rewrite the context available in the self object by adding the new key:value pair you have mentioned. There are times, this might not work as expected, and you want a patch technique to update the context. This can be done by changing the data type to dict.

self.env.context = dict(self.env.context)
self.env.context.update({
'test': 'test_value',
})

This would also add the new key:value pair to your context and work as expected. There is almost no security concern here for converting the frozendict as the context will destroy once the page is left soon enough.

How to Validate Odoo Invoice Automatically

You might want to validate your odoo invoices automatically, without bothering your users to go to each vendor/customer bill and validate them, instead triggering based on certain conditions. It’s a 4 step code:

First, we get the invoice:

invoice_id = self.env['account.invoice'].browse(your_invoice_id)

Assign the date to it:

invoice_id.action_date_assign()

Create the invoice moves:

invoice_id.action_move_create()

Now you can validate the invoice:

invoice_id.invoice_validate()

How to Add Invoice to Purchase Order in Odoo Automatically

I was facing an issue in two way matching algorithm for Incoming Receipts & Vendor bills in my Odoo installation. As vendor bills are tagged in Purchase Orders only through use of ‘Create Bill’, I had to make an algorithm to calculate the matching. Remember, the match would only work if the supply chain members follows the right way of doing it, otherwise, it would fail. That’s what happened, it fails almost 3 times out of 10. Then I realized, the invoice needs to be pinned automatically in the stock.picking model, instead of purchase.order model at the time a Receipt is validated, to keep the tracking for my matching.

While doing so, first challenge is to pin an invoice to a purchase id. By default, the model for invoicing, ‘account.invoice’ has a foreign read only key for purchase, which is purchase_id. But remember, this is a read only field, and can’t be written/created in a new or existing record using record. I dig down the Odoo codes a little under purchase/models/account_invoice.py and could find the Odoo function that can do the job for us. It’s called ‘purchase_order_change()’.

Here is a snippet of adding an invoice automatically to a purchase ID:

invoice_id = self.env['account.invoice'].create({
'type': 'in_invoice',
'purchase_id': self.purchase_id.id,
'partner_id': self.purchase_id.partner_id.id,
})

invoice_id.purchase_order_change()

This will create an invoice in Draft state. Remember, you need to validate this invoice, if you would like to create the invoice number, not just an invoice entry. You may do it manually or you may do it automatically ‘How to Automatically Validate an Invoice in Odoo

Remember, I am triggering the snippet from the stock.picking, not from purchase.order, which is why, the ‘self’, is related to stock.picking and calling self.purchase_id to link the original purchase_id that triggered the stock.picking record.

Now, a small tip. What if, if you want to align the invoice to only have the products you have received in this particular receipt? Easy as pie:

for invoice_line in invoice_id.mapped('invoice_line_ids'):
check = 1
for move_line in self.mapped('move_line_ids'):
if invoice_line.product_id.id == move_line.porduct_id.id:
check = 0

if check:
invoice_line.unlink()

Simple! Isn’t it? Happy coding!

How to hide a column ‘dynamically’ in Tree View on Odoo

Odoo field has an attribute called ‘attrs’. This adds custom attribute to the Odoo form/tree views. You may use different kind of attributes with attrs like ‘required’, ‘readonly’ or ‘invisible’. For basic, to hide a field regardless of it be an individual entity or parent relation, we use the following:

<field name='field_name' invisible='1'/>

This is useful when we are triggering a computed field that we don’t want to show on a tree/form view. But what if we want to dynamically decide whether to invisible the field or not? For such cases, we usually use invisible with the ‘attrs’ attribute on fields, for example:

<field name='product_id' attrs="{ 'invisible': [('is_set', '=', True)]}"/>

You might set the field 'is_set' itself as invisible and a boolean computed field like the following:

is_set = fields.Boolean(compute="_set_is_set")

def _set_is_set(self):
if self.lot_id.life_date:
self.is_set = True
else:
self.is_set = False

Seems easy, right? But there is a catch, the above won’t hide the whole column if you are on a place like ‘purchase.order.line’ or ‘stock.move.line’. How can you work on such cases? It only works on individual entry, but not on a relational column. There is an attribute called ‘column_invisible’ same like ‘invisible’ we have used above. But the difference is, you need to set this based on parent value, thus hides a relational column. Here is an example to hide a column called ‘show_expiry’ in a picking operation:

<field name="show_expiry" attrs="{ 'column_invisible' : [('parent.picking_type_code', 'in', ['incoming'])]}" />

Above code checks the ‘picking_type_code’ value from ‘stock.picking’ model, and hides the column if it’s an incoming shipment and shows when it’s an outgoing shipment. That means, the above, would show the column (show_expiry) if it’s a Delivery to customer location or you already have the product, but won’t show if you are doing a GRN, means you don’t have the stock yet, just arriving.

Pretty simple, isn’t it? Good luck.