New IFC optimiser tool

M by Moult on 4 Aug 2020, edited 6 Mar 2024 # +6 votes

It's been on my to-do list to write a free software equivalent of something like Solibri Optimiser. I'm not sure what the secret sauce is, but the logical approach towards IFC optimisation seems to simply recycle non-rooted elements.

It turns out you can do this in 10 lines of code (excluding IfcPatch boilerplate): https://github.com/IfcOpenShell/IfcOpenShell/commit/61f6ecbf179bae1eca6baef378c246a35100d414

I tested it on an IFC from the electrical engineering discipline (arbitrary, had it lying around) and it was 119,372kb prior to optimisation (direct from Revit), and it became 89,960kb after lossless optimisation.

In short, 25% of the file size could be safely removed.

Importing the unoptimised one in Blender took 103.26 seconds, and the optimised import took 94.76 seconds.

I know that FreeCAD has a recycler on export-time (BlenderBIM Add-on does not, so I expect IFCs from Blender to be rather inefficient too), so it'd be interesting to run this on a large FreeCAD IFC.

Just for fun, I zipped it (for .ifczip) and it ended up being 19MB. Not too shabby for a large project. Tiny filesizes indeed!

M by Moult on 5 Aug 2020, edited 5 Aug 2020 # +1 votes

I tested the same file with Solibri optimiser and got 81,696kb.

But the way this optimiser works is that you can choose how many iterations you want to recycle elements. If you run it more than once, you optimise it further, with diminishing returns. So, I ran it a few more times and got 80,356kb. Woo!

Because Solibri's optimiser is a black box, I don't know these for sure, but here's my attempt at describing the differences:

Solibri's optimiser seems to produce smaller results if you only run it once
IfcPatch optimiser seems to produce smaller results if you iterate
Solibri's optimiser is faster.
Solibri's optimiser mangles STEP IDs. This may be important if you're doing some scripty voodoo, or tying together datasets for viewing on the web, for example. IfcPatch optimiser preserves STEP IDs.
I suspect that some non rooted elements cannot by recycled without side effects. For example IfcProductDefinitionShape in IFC4 has a 1:N mapping, but in IFC2X3 is 1:1, so if you recycle it, technically you're breaking some IFC2X3 rule (admittedly it wasn't an important rule, but some vendors might rely on it). I think Solibri guards against this, whereas IfcPatch will break it. It sounds like a pretty boring job to check these rules, so I'm ignoring it for now.
Solibri's is available on Windows and Mac. IfcPatch is fully cross platform: Windows, Mac, Linux.
I didn't see a way to run Solibri's one headlessly, could've missed something though. IfcPatch can be headless. Or even used as a library in your own app.

B by brunopostle on 5 Aug 2020 #

In my tests doing graph sorting of IFC files, if you renumber the step entities such that those that are referenced a lot get lower numbers, this itself can reduce the files by 5% - simply because #12 uses less bytes than #34567

M by Moult on 5 Aug 2020 #

@brunopostle good point - not sure off the top of my head how to do that efficiently, but I'll ponder it :)

B by bernd on 26 Feb 2021 #

ifcpatch.execute({

    "input": "/home/hugo/NeuOrd/IFC Schependomlaan_remove.ifc",

    "output": "/home/hugo/NeuOrd/IFC Schependomlaan_NonRootedRemoved.ifc",

    "recipe": "RecycleNonRootedElements",

    "log": "ifcpatch.log",

    "arguments": "????",

})

What do I need to put in "arguments" to delete all non rooted element in a ifc?

B by bernd on 26 Feb 2021 #

What is the difference between the recipes "Optimise" and "RecycleNonRootedElements"?

B by bernd on 26 Feb 2021, edited 26 Feb 2021 #

it seams:

RecycleNonRootedElements does what it says
Optimise remappes duplicates (https://github.com/IfcOpenShell/IfcOpenShell/blob/3eacd809b0d/src/ifcpatch/ifcpatch/recipes/Optimise.py#L30)

B by bernd on 26 Feb 2021, edited 26 Feb 2021 #

the following works great here:

ifcpatch.execute({

        "input": "/home/hugo/NeuOrd/IFC Schependomlaan_remove.ifc",

        "output": "/home/hugo/NeuOrd/IFC Schependomlaan_NonRootedRemoved.ifc",

        "recipe": "RecycleNonRootedElements",

        "log": "ifcpatch.log",

        "arguments": [],

    })

B by bernd on 26 Feb 2021, edited 26 Feb 2021 #

What I did ...

get some big ifc sample data from here https://github.com/buildingSMART/Sample-Test-Files/tree/master/IFC%202x3/Schependomlaan/Design%20model%20IFC
47.0 MB
remove all but not IfcStair


import ifcopenshell

ifc_in = "/home/hugo/NeuOrd/IFC Schependomlaan.ifc"

ifc_out = "/home/hugo/NeuOrd/IFC Schependomlaan_remove.ifc"

f = ifcopenshell.open(ifc_in)

products_to_remove = [p for p in f.by_type("IfcProduct") if not p.is_a("IfcStair")]

for x in products_to_remove:

    f.remove(x)


f.write(ifc_out)

But this only removes the product entities, but not the geometry or the material from the removed products, thus I tried "RecycleNonRootedElements". Means the file is still 45.4 MB


import sys

sys.path.append("/home/hugo/Documents/dev/ifcopenshell/ifcopenshell-official/ifcos/src/ifcpatch/")

import ifcpatch


ifcpatch.execute({

    "input": "/home/hugo/NeuOrd/IFC Schependomlaan_remove.ifc",

    "output": "/home/hugo/NeuOrd/IFC Schependomlaan_NonRootedRemoved.ifc",

    "recipe": "RecycleNonRootedElements",

    "log": "ifcpatch.log",

    "arguments": [],

})

This removes a lot. The file is 22 MB, but all geometry is still in there ...

Example:

#809= IFCBEAM('2501ntWb95m8MEJo4qrYny',#25,'fund_opstort',$,'IFC_betonopstort_220x315 220 x 315',#747,#805,'85001C77-8252-45C0-858E-4F2134D62C7C');

geometry:

#805= IFCPRODUCTDEFINITIONSHAPE($,$,(#797,#803));

Which still completly with all its childs in the file after run "RecycleNonRootedElements"

material

orginal:

#813= IFCRELASSOCIATESMATERIAL('3hUI20KmNDbknDCziP_kse',#25,$,$,(#809),#482);

after remove and RecycleNonRootedElements:

#813=IFCRELASSOCIATESMATERIAL('3hUI20KmNDbknDCziP_kse',#25,$,$,(),#482);

this should be removed as it does not reference a product any longer.

B by bernd on 26 Feb 2021, edited 26 Feb 2021 #

@bernd said:

it seams:

RecycleNonRootedElements does what it says

Just played a bit an I was totally wrong with my assumption what RecycleNonRootedElements does. I may miss what a NonRootedElement is. Looking at the code RecycleNonRootedElements does delete duplicate ifcelements (lines) out of the ifc.

RecycleNonRootedElements would I understand to delete dead elements. Elements which are no longer used (referenced) anymore. For example a geometry (IfcProductDefinitionShape) from a removed BuildingElement.

M by Moult on 27 Feb 2021 # +1 votes

A non-rooted element is any IFC class that doesn't inherit from IfcRoot. It was my attempt at optimisation. It probably has some issues, but note that it can run multiple times on the same file, each iteration recycling more and more.

B by bernd on 28 Feb 2021, edited 28 Feb 2021 #

@Moult

Do you know some python code to remove the whole dead (no longer linked) geometry of a object if the object was removed by ifcfile.remove(element) ?

M by Moult on 28 Feb 2021 # +1 votes

@bernd yes. Removing a representation is not a "simple" task. You need to consider things like mapped representations, multiple representations, shared non-rooted elements, presentation layers, and styled items. Easy to disconnect, hard to purge correctly without experience in the spec. I'll try to explain it as best I can, but keep in mind this is complicated.

First, a more general tool that helps is the ifcopenshell.util.element.remove_deep() function. It purges elements down the tree that aren't used by other elements outside that branch. It's good enough for purging a lot of things - but not enough for purging geometry.

The good news is that the BlenderBIM Add-on code is getting more and more agnostic of Blender - and you can reuse it to manipulate IFCs without Blender, and all the hard work has been done for you.

If you want to remove the entire element, this is the code you need to run: https://github.com/IfcOpenShell/IfcOpenShell/blob/v0.6.0/src/ifcblenderexport/blenderbim/bim/module/root/operator.py#L171-L199 - let me know if it makes sense or not. Once you strip away the Blender stuff, this is the agnostic code you need to run.


product = self.file.by_id(12345) # This is your product, e.g. IfcWall, and self.file is your IfcOpenShell file object

for representation in product.Representation.Representations:

    unassign_representation.Usecase(self.file, {"product": product, "representation": representation}).execute()

    remove_representation.Usecase(self.file, {"representation": representation}).execute()

remove_product.Usecase(self.file, {"product": product}).execute()

The relevant usecases are here:

W by wmi on 5 Mar 2024 #

I just tried the 'optimise' recipe and ran into an error concerning this line "from toposort import toposort_flatten as toposort". It happened on a windows 10 machine using blender 4.0.2 with version 0.0.240305 of the BlenderBIM Addon. The app complains that there is no module named toposort to be had. Am I missing another library?

A by Andrej730 on 6 Mar 2024 #

@wmi it indeed wasn't included in BBIM yet, you can install it from BBIM Debug section:

W by wmi on 6 Mar 2024 #

@Andrej730 The debug section in my installation does not provide a Pip Install feature.

A by Andrej730 on 6 Mar 2024 # +2 votes

@wmi oh, sorry, it was added just today, you'll need to download new BlenderBIM build