OSArch Community

CSV Export from multiple models | future patch?

  1. S

    introduction

    there are a few models from which I need to extract properties and quantities, like when I link multiple ones in a federated arrangement

    if it was for one model only I would use Spreadsheet Import/Export

    combining multiple models requires some script

    thankfully ChatGPT helped me out in putting this together, so if any programming bad style it's its fault..

    modules/functions used:

    import modules and

    1. Load IFC models into a list of dictionaries
    
    import ifcopenshell
    
    from ifcopenshell.util.selector import filter_elements
    
    from ifccsv import IfcCsv
    
    import os
    
    def load_model_list(model_paths: list[str]) -> list[dict]:
    
        return [
    
            {
    
                "model": ifcopenshell.open(path),
    
                "name": os.path.splitext(os.path.basename(path))[0],
    
                "path": path
    
            }
    
            for path in model_paths
    
        ]
    
    1. Export filtered elements from a model to a CSV in the same folder
    
    def export_model_to_csv(model, model_name: str, model_path: str, query: str, attributes: list[str]) -> None:
    
        elements = filter_elements(model, query)
    
        if not elements:
    
            print(f"⚠️ No matching elements in model '{model_name}'")
    
            return
    
        model_dir = os.path.dirname(model_path)
    
        output_file = os.path.join(model_dir, f"{model_name}_extract.csv")
    
        ifc_csv = IfcCsv()
    
        ifc_csv.export(
    
            model,
    
            elements,
    
            attributes,
    
            output=output_file,
    
            format="csv",
    
            delimiter=",",
    
            null="-"
    
        )
    
        print(f"✅ Exported model '{model_name}' to:\n   {output_file}")
    
    1. Main function

    change model_paths, query, and attributes to your need

    in this example query and attributes select all IfcWall elements with type starting with "WAL-"

    from selected elements extracts attributes like Name, type.Name, and NetSideArea

    before running the script make sure that original models have quantity take-off already done

    
    def main():
    
        model_paths = [
    
            r"path to ifc file\filename1.ifc",
    
            r"path to ifc file\filename2.ifc"
    
        ]
    
        query = 'IfcWall, type=/WAL-.*/' 
    
        attributes = ["Name", "type.Name", "Qto_WallBaseQuantities.NetSideArea"]
    
        models = load_model_list(model_paths)
    
        print("\n📂 Loaded Models:")
    
        for m in models:
    
            print(f"- {m['name']}: {m['path']}")
    
        for m in models:
    
            export_model_to_csv(
    
                model=m["model"],
    
                model_name=m["name"],
    
                model_path=m["path"],
    
                query=query,
    
                attributes=attributes
    
            )
    
    1. Run the script
    
    if __name__ == "__main__":
    
        main()
    

    Code attached

    I would hugely appreciate your feedback, thanks

  1. F

    @steverugi Here is a possible PR Multiple ifc spreadsheet export #6548 to enhance the exporting capabilities of Quality & Coordination -> Spreadsheet Import/Export.

    This is in line with Issue allow multiple .ifc file in Spreadsheet Import/Export, or linked ones #6252

    It allows to select a number of files (no error checking yet) below the loaded ifc file directory structure. In the case of web output it concatenates the results. In the case of CSV it generates a file in the same directory adding the date-time suffix and the file format extension.

    Thanks!

  1. S

    @falken10vdl

    wow! that's cool, thank you so much

    only one thing if you like, the file selection I had in mind was similar to the one used in "Link IFC", at first impression looks more compact and allows individual selection of the files, also showing their path

    !

    but I'll be happy with any way the community wants to have it

    really a useful feature, thanks again for your help

  1. F

    @steverugi You are right. Probably best practice is as you say. Let me change it to follow your advice.

    BTW, I was doing that tree structure because I was playing with the idea of some sort of "Project Management Panel/checklist". I have come to know that the IFC file is just a portion of a complete IFC based project. That is why I had the the idea of having a tool that will allow you to interact with all the files involved in the project so then it is simple to pack (zip) and provide to someone without losing information due to absolute paths or blender/bonsai "internal" directories. Also the links in relative paths so it can be loaded somewhere else.

    Ex:

    !

    Example the "IFC Project Files Management" Provides details below the IFC main file. You can see that it shows the linked files and puts a Triangle icon for the case that is an absolute path (bad idea to transport), a hand if it is a relative path (good idea but the relative shoudl be below the main IFC file so a zip file package can easily be created) or a link if it is ok. So it is a sort of a checklist tool with good practice in order to make a whole package that you can deliver to someone else (probably info about blender/Bonsai/Sun/etc extensions should also appear)

    What do you guys think about this "IFC Project Files Management" idea. Is this something you find interesting?

    Thanks!

  1. S

    @falken10vdl

    BTW, I was doing that tree structure because I was playing with the idea of some sort of "Project Management Panel/checklist". I have come to know that the IFC file is just a portion of a complete IFC based project. That is why I had the the idea of having a tool that will allow you to interact with all the files involved in the project so then it is simple to pack (zip) and provide to someone without losing information due to absolute paths or blender/bonsai "internal" directories. Also the links in relative paths so it can be loaded somewhere else.

    Example the "IFC Project Files Management" Provides details below the IFC main file. You can see that it shows the linked files and puts a Triangle icon for the case that is an absolute path (bad idea to transport), a hand if it is a relative path (good idea but the relative shoudl be below the main IFC file so a zip file package can easily be created) or a link if it is ok. So it is a sort of a checklist tool with good practice in order to make a whole package that you can deliver to someone else (probably info about blender/Bonsai/Sun/etc extensions should also appear)

    What do you guys think about this "IFC Project Files Management" idea. Is this something you find interesting?

    I like the idea of having a "Project Management Panel/ checklist", if possible it should be in a pop-up window, similar to what we use to import a file for instance. I am not sure how the same can be preserved in a full IFC environment though (I guess the same way we save Link paths..)

    In general I tend to like consistent processes across the board, that's why my proposal to implement a "Link IFC" - like approach.

    I am looking forward to seeing this feature implemented, hopefully more users will add more feedback to your video

    Thanks again, cheers

  1. B

    @falken10vdl said:

    What do you guys think about this "IFC Project Files Management" idea. Is this something you find interesting?

    Something like this would be useful for managing the non-ifc files in a git repository

  1. F

    Here a solution using UILists like in other parts of Bonsai

    Do you know what the Generate SVG and Preserve Existing are supposed to do?

    !

    Probably another setting for "Performing Quantity Take off" would make sense? As another icon to check per file?

    Thanks!

  1. M

    @falken10vdl i think tree structure looks very nice but visually it needs more space than the UIList so i prefer the second one, also for consistency. It would be good also to have the possibility to select multiple files at the same time.

    Btw, the tree structure is very interesting, i'm sure it is useful for some purpose...maybe for managing linked documents or attached images?

  1. F

    @Massimo I agree. Let me add the multiple select file possibility. Since it is using the builtin file explorer I think that you can only select multiple files from the same directory right?

    Thanks!

  1. S

    Hi @falken10vdl

    I don't want to spoil the game but can we just start with a simple panel like in "Link IFC"? There you have the path and option icons on the right. (Also useful for selection)

    Additionally, if not too difficult, it would be handy to have a box to "select linked models" to operate in federated mode

    Thanks a lot for your help

  1. M

    @falken10vdl yep, i think so

  1. S

    @falken10vdl said:

    Do you know what the Generate SVG and Preserve Existing are supposed to do?

    no idea, tried if it did something but couldn't find anything, maybe looking at the code it can be found out?

    can you share the link to the code please?

    Probably another setting for "Perform Quantity Take off" would make sense? As another icon to check per file?

    mixed feelings about this, there are already two in the current UI

    • Object Metadata > Quantity Sets

    • Costing and Scheduling > Quantity Take-off

    both of them with qto engine selection, wouldn't it be a bit confusing? sometimes you need to use either of them to capture specific quantities, or need manual quantification for particularly difficult ones or full manual entry.

    I'd be careful before adding another one, my POV of course

  1. F

    @steverugi A simple way to see the code is if you have enabled Edit->Preferences->Interface->Display->Developer Extras then you can right click in the pannel and select "Edit Source"

    !

    Then in Scripting you can see the relevant code:

    !

    It will actually mark the point in the code:

    !

    From there I find quite usefull to search in VSCode over the file directory:

    !

    The you click in it and there you have it:

    !

    Let me add this "tip" to the PR (https://github.com/IfcOpenShell/IfcOpenShell/pull/6494) I submitted in https://community.osarch.org/discussion/comment/24629#Comment_24629

    Thanks!

  1. F

    Ok. I have updated the panel to look like this:

    !

    There is a button "Add Linked Files" that will fetch them and add them to the list.

    Each entry in the list has three options to select:

    1. Tickmarck to select the file for exporting,

    2. "eye" to open the file in a new blender instance (instead of doing Qty take off in settings, as per @steverugi suggestion, I have added this icon that could be usefull if one wants to check and save the file so it is ready for the csv/web export)

    3. "cancel" to remove the entry

    One can use both absolute and relative paths.

    Please let me know if this is good enough for a PR or something else is missing

    Thanks!

  1. S

    @falken10vdl

    good enough? it's perfect! :D

    1. "eye" to open the file in a new blender instance (instead of doing Qty take off in settings, as per @steverugi suggestion, I have added this icon that could be useful if one wants to check and save the file so it is ready for the csv/web export)

    for me to understand, does it open a new Bonsai session for potential editing in case of some change like QTO or else? would it be not simpler to adopt the "sync" icon Massimo activated no long ago in "Link IFC" ?

    thank you so much for your help (and patience)

  1. F

    @steverugi cool! Thanks for yor input. Yes it is as you say, it triggers a new process with an independent blender instance and automatically loads the ifc project so one can edit and make whatever adjustments.

    @Massimo how does the "sync" icon work? If I take "load from.memory" for example, the export will dig into the items linked in the main file?

    Thanks!

  1. M

    @falken10vdl well, nothing special, it unlink and relink an ifc linked model so it can be updated "on the fly", look at here (https://github.com/IfcOpenShell/IfcOpenShell/blob/e8b74245ce8941f310c7c28c700129b79114dccc/src/bonsai/bonsai/bim/module/project/operator.py#L1427)

    I think that "a new process" (with eye icon) and "sync" (refresh) have different purposes: the first one could be useful if i want to open the model in a new dedicated window and the second one could be useful if i want to stay in the same session and update the file without open it.

    So, maybe, the "eye" icon is useful under the panel Project setup -> Links near the existing ones icons (Reload link, Unload link, etc...) and the "sync" icon is useful in the Spreadsheet import/export panel...no? What do you think?

    Btw, the work you have done is great!

  1. T
  1. S

    @Massimo & @falken10vdl

    @falken10vdl well, nothing special, it unlink and relink an ifc linked model so it can be updated "on the fly", look at here (https://github.com/IfcOpenShell/IfcOpenShell/blob/e8b74245ce8941f310c7c28c700129b79114dccc/src/bonsai/bonsai/bim/module/project/operator.py#L1427)

    I think that "a new process" (with eye icon) and "sync" (refresh) have different purposes: the first one could be useful if i want to open the model in a new dedicated window and the second one could be useful if i want to stay in the same session and update the file without open it.

    So, maybe, the "eye" icon is useful under the panel Project setup -> Links near the existing ones icons (Reload link, Unload link, etc...) and the "sync" icon is useful in the Spreadsheet import/export panel...no? What do you think?

    I was thinking:

    • do we really need a dedicated icon (eye) to open a project we need or want to edit in a different session? can't we just do it outside of that session (e.g. start new Blender/Bonsai and open file)

    • I find extremely useful to query multiple models, especially with the option where they are the current and linked ones, and generate a common schedule, but the "sync" (which I proposed earlier) can easily be the one from the "Link IFC" panel

    • to test the new feature I would just start with indispensable options, to avoid overpopulated UI that may disorient new users

    Btw, the work you have done is great!

    Super great!

  1. S
  1. T
  1. S

    @theoryshaw

    a little fishing tour. :)

    little help please, where is the 'svg' going to be saved? thanks

  1. T

    in the same folder the .ods file is saved.

  1. S

    thank you @theoryshaw

    the final arrangement of the columns in the .svg file needs some help

    !

    I opened it in Inkscape and the text is there but overlapping,

    Adjusting the column width in .ods > save it > run it again with "preserve existing" triggers an error

    Better to export it from .ods maybe, unless I missed something

    Cheers

    PS why only .ods has the .svg option?

  1. T

    ¯\(ツ)

Go to page:

  1. 1
  2. 2

Login or Register to reply.