GSoC 2024 call for project ideas!

M by Moult on 22 Jan 2024, edited 4 May 2024 # +4 votes

Google Summer of Code is a global, online program focused on bringing new contributors into open source software development. GSoC Contributors work with an open source organization on a 12+ week programming project under the guidance of mentors.

As of today, applications for organisations are open (not yet for contributors). I expect projects like BRL-CAD et all, IfcOpenShell / BlenderBIM Add-on, FreeCAD, Blender, GIMP, OpenStreetMap, OSGeo, CGAL, who all joined in last year to potentially also be available this year.

Here is the list of ideas of projects for IfcOpenShell / BlenderBIM Add-on for 2024: https://github.com/opencax/GSoC/issues?q=is%3Aissue+is%3Aopen+label%3A%22Project%3A+IfcOpenShell%22

Note that these are just ideas. You can choose from the list of ideas, or come up with your own idea! Anybody who is new to contributing to open source can join! Do you have an idea? Now is a good time to add it!

FreeCAD also has a list of ideas: https://wiki.freecad.org/Google_Summer_of_Code_2023 - if you are a FreeCAD user or dev and have an idea, now is a great time to add it (and start a new page for 2024 potentially).

N by Nigel on 22 Jan 2024 # +4 votes

Before posting something silly or redundant on IfcOpenShell / BlenderBIM Add-on for 2024 GitHub I wanted to run it by the smart people here on the forum. My idea is this: Paper to IFC maker

Outline

There is a mountain of ‘paper plans’ of existing buildings around the world, energy analysis, carbon assessment and maintenance forecasting are just a few areas where a portfolio of IFCs would be helpful. I propose turning these paper plans into IFC with the minimum of human intervention.

With the use of AI, the paper plan can be analyzed; walls, windows, doors, stairs etc identified through their typical graphical representation and a suitable IFC class/representation substituted in a new IFC file in the BlenderBIM Addon.

Human intervention will likely be required to establish/confirm drawing scale and may include some written notes, symbols, room names or window and door numbers for example. This could be read by the AI and used to define spaces and name IFC components. Window and door numbers could even be used to pick objects from an IFC openings library.

Details

Training the AI would include an ability to train for new drawing styles, this could be done through adding snips of graphical representations of elements such as walls, windows, doors, stairs etc to a reference library. Default settings like wall thickness and ceiling heights can be set. The AI would return a ‘confidence’ assessment of its work so a human can check and confirm or correct any mistakes. This stage would also feed back into AI learning. Drawing styles can be saved separately for future use.

Expected Outcome

The process of converting a paper plan to a usable IFC would be simplified and quick.

Human intervention would be coaching and confirming through knowledge and experience, reducing tedious modelling.

This would leverage the BlenderBIM addon to create IFCs.

Measurement and analysis of existing buildings would be greatly enhanced through digitizing.

Other Possibilities

The Paper to IFC maker could also be used to turn traditionally hand drafted concept designs into IFCs allowing a person with no BIM skills to participate in BIM. This would leverage the skills of often older, highly skilled designers to efficiently be involved more directly in BIM projects.

B by bdamay on 23 Jan 2024 #

@Nigel

I love your idea! i spent so many hours modelling existing buildings from paper plans, that, at that time, i would have loved to have a tool like this.

Let me share some thoughts about this (that i hope are not too negative)

I wonder if there is an initial set of validated dataset available to train a AI model on ? some kind of paper2ifc association set that in my sense is needed if we hope to train an AI model. Or may be you think AI has to (may be) focus on elements ( something like "yes this line comes from a wall not a slab boundary, nor a leader arrow"). I've seen so many different kinds of drawings that i wonder if AI could really deal efficiently with this kind of stuff. (May be reduce the scope to first analyse vectorized 2D plans would be a good first step, i don't know... )

That said, I could imagine a UI feature that guide the user to help human by AI proposing value for a portion of drawing, and train AI with the help of human validating these portions of the plan and its IFC modelisation with the human look. A bit like autocompletion does in many tool we daily use. In practice i have usually modeled from plans and photos, not only plans.

Finally, if the purpose is to represent reality, i wonder if the best effort to be made is not to put AI effort on the analysis of 3D scan to create 3D ifc models. 3DScan seems to be really cheaper and simpler to do today. (A friend told me last month that he as a lidar in is iphone !).

Plans are good, but sometimes they are not correct, they are old, come from different period of time, different architects and i don't expect AI to be really able to deal with that complexity.

I might be wrong... i'd love to be wrong ... please prove me wrong !

Wish you a good day

M by Martin156131 on 23 Jan 2024 # +3 votes

Last summer, I visited a conference where young people presented their findings on this topic, paper2ifc:

Method for generating multimodal synthetic data from parametric BIM models for use in AI systems:

https://hss-opus.ub.ruhr-uni-bochum.de/opus4/frontdoor/index/index/searchtype/collection/id/17165/start/7/rows/10/facetNumber_subject/all/docId/10130

Floorplan2IFC: Transformation of building floor plans into hierarchical IFC-graphs

https://hss-opus.ub.ruhr-uni-bochum.de/opus4/frontdoor/index/index/searchtype/collection/id/17165/start/15/rows/10/facetNumber_subject/all/docId/10122

Towards the rule-based synthesis of realistic floor plan images

https://hss-opus.ub.ruhr-uni-bochum.de/opus4/frontdoor/index/index/searchtype/collection/id/17165/start/48/rows/10/facetNumber_subject/all/docId/10089

Towards automated digital building model generation from floorplans and on-site images

https://hss-opus.ub.ruhr-uni-bochum.de/opus4/frontdoor/index/index/searchtype/collection/id/17165/start/5/rows/10/facetNumber_subject/all/docId/10132

And all other papers can be found here:

https://hss-opus.ub.ruhr-uni-bochum.de/opus4/solrsearch/index/search/searchtype/collection/id/17165/start/50/rows/10/facetNumber_subject/all

N by Nigel on 23 Jan 2024 #

This is not AECO specific but does speak to the idea of manual mark-ups and AI interpretation and conversion to digital, cookies are important too! some silly online 'clickbait' headline I saw today said 'Two-thirds of Americans said AI could do their jobs' at that rate 6 billion of us worldwide will be looking for work

M by Moult on 23 Jan 2024 #

That's a really cool idea! GSoC ideas need to have a mentor, do you know anybody with AI/ML knowledge that would be able to mentor that aspect of it?

N by Nigel on 24 Jan 2024 #

@Moult said:

That's a really cool idea! GSoC ideas need to have a mentor, do you know anybody with AI/ML knowledge that would be able to mentor that aspect of it?

Not me I'm afraid, but I would love to be involved as tester, reviewer or whatever that role is called

B by buovjaga on 24 Jan 2024, edited 24 Jan 2024 # +1 votes

@Moult said:

Here is the list of ideas of projects for IfcOpenShell / BlenderBIM Add-on for 2024: https://github.com/opencax/GSoC/issues?q=is%3Aissue+is%3Aopen+label%3A%22Project%3A+IfcOpenShell%22

Note that when applying for GSoC, you should never point to a list of issues as the ideas page. Using such a link will quite certainly lead to a rejection. If an application has already been made, I would suggest to prepare a proper page and request to edit the application.

Reference: Defining a Project (Ideas List) .

Never link to just a bug tracker. This is almost always an automatic rejection. Yes, really. When Admins open the issue tracker if there is an

expanded description of the project with the items listed in #1 above that is acceptable. If an org can’t take the time to create a document or

a page with their Ideas List fleshed out then why would Google expect they would be willing to mentor GSoC contributors for 3+ hours a

week for the next 3+ months?

B by bdamay on 25 Jan 2024 # +3 votes

Hi, i thought further on @nigel proposal, and took a closer look to @Martin156131 papers, and more and more, i feel like working on this subject. I don't know if it's GsoC compatible, but here is the thing:

I am actually following a course on Datascience and ML at https://datascientest.com/ in Paris.

It's sort of a Master degree in Data analysis- data science, and Machine learning processes, a bit of MLOPs as well.

In the 2 or 3 next weeks, i have to pick a ML project to work on, what they call the "Projet fil rouge" and this is a important part of what i have to achieve to validate my degree.

There is a list to choose from, but i also have the opportunity to propose my own subject, if it fills the requirements needed.

The project ends in october, and is mainly cut in 3 parts.

Data analysis Data visualisation - data cleaning - data preprocessing
Data modeling - training data with regression models and/or neural networks on the data to make predictions -
Presenting results on a interactive web app like streamlit or equivalent

For this proposal to be accepted by my mentor, i first need to prove i am able to access sufficient training data for the model and that the subject is relevant. For the data part, in my understanding, i would need paper plans, sections, elevations, and their corresponding ifc as "training data".

It doesn't have to be perfect data, I'll be judged on the ability to handle the ML process more than on the results (But sure good results would be more satisfying)

That is why i ask you, dear community, if you know where i could find that kind of data? open data ? If it's a concern, i'm ready to sign any NDA, if needed (and i guess my mentors will accept as well ).

If you have any other insight to share with me, please do. I would be delighted to combine my ML training journey on a subject i like (BIM) rather than predicting who might buy a new car next month.

Thanks for your help.

B by Bimlooser on 25 Jan 2024 #

@bdamay said:

Finally, if the purpose is to represent reality, i wonder if the best effort to be made is not to put AI effort on the analysis of 3D scan to create 3D ifc models. 3DScan seems to be really cheaper and simpler to do today. (A friend told me last month that he as a lidar in is iphone !).

Plans are good, but sometimes they are not correct, they are old, come from different period of time, different architects and i don't expect AI to be really able to deal with that complexity.

I'd like a lot something about scan2ifc

T by theoryshaw on 25 Jan 2024 # +2 votes

That is why i ask you, dear community, if you know where i could find that kind of data? open data ?

You're free to use any pdfs in any of the repos under the following. All open source.

Obviously there's just one 'drawing style' represented on most of the pdfs, but maybe a good start for training Skynet. :)

B by bdamay on 25 Jan 2024 #

thanks @theoryshaw, didn't know about these, i'll look into it tomorrow.

T by theoryshaw on 25 Jan 2024 #

NP. Some of the most recent repos have PDF's from BB/IFC files. I'm no AI engineer, but I wonder if there's any advantage of training the AI by comparing both files together?

T by theoryshaw on 25 Jan 2024, edited 25 Jan 2024 # +1 votes

Also, i'm no software engineer, but my gut feels like you might use the following as a foundation...

https://github.com/opensourceBIM/voxelization_toolkit

M by Martin156131 on 26 Jan 2024, edited 26 Jan 2024 #

The first paper I linked to is about how difficult it is in the construction industry to get large data sets to train an AI model. Even if you have these large data sets, for example PDF plans, you still have to “label” them, and doing this manually is too time-consuming. The solution chosen here is to use a script to create 2D PDF plans from a randomly generated 3D BIM model and to use the 3D BIM model to “label” them correctly using the script. This allows any number of synthetic, “labelled” data to be generated that are relevant for training an AI model. This was done in that Paper with the Revit API, I will find it exciting to see if you can create such “labelled” data with the same approach but with the BlenderBIM API.

B by bdamay on 26 Jan 2024, edited 26 Jan 2024 # +1 votes

@Martin156131, i confess i didn't read the paper fully, if i remember well, it's german, and my german is quite poor (guilty!)

I had a meeting this morning with my mentor, and unfortunately, we both agreed the case study seems too complex to handle at my level of expertise. I will have to find something a bit less ambitious to deal with. So i'm now thinking about detecting and setting up roof shapes from satellite photos to push a bit further the process of setting up 3D environnement contexts i use, as i show in the video below.

I have found papers like https://www.researchgate.net/publication/328972501_Roof_Shape_Classification_from_LiDAR_and_Satellite_Image_Data_Fusion_Using_Supervised_Learning but don't really know at what stage it is.

Do you think it might be an idea of value?

T by theoryshaw on 26 Jan 2024 #

I'd like a lot something about scan2ifc

I 2nd this, as well.

J by JanF on 26 Jan 2024 #

So you are abandoning this? I managed to get some promising contacts of the ML experts at the Prague CVUT and could try to find somebody there to mentor that part.

B by bdamay on 26 Jan 2024, edited 26 Jan 2024 #

@JanF said:

So you are abandoning this? I managed to get some promising contacts of the ML experts at the Prague CVUT and could try to find somebody there to mentor that part.

@JanF. Guess you are talking to me ? If yes, i would say, no i'm not abandoning it fully yet, i'm still interested to work on the subject, and any if your help is very nice. What i am sure of is that i will not be able to make it the "Projet Fil rouge" for my degree. I'm on the first months of my ML journey and starting with this might be a bit too difficult for me to handle. I agreed with my teacher with that.

However, for a starting tip, he advised me to use IFC model to create plans and use these pairs Plan/IFC as "good data" we can feed the model with. This before trying to go anywhere further. Think it is a good idea to start with.

He also pointed this YOLO stuff out for me (see video below)- it's quite impressive. i wonder if this can be used for Arch purposes .

For my "Projet Fil Rouge", my fall back plan is to try out a simpler case wich would be analysing satellite photos to guess roof shapes. as i said earlier.

B by bdamay on 29 Jan 2024 #

@theoryshaw said:

I'd like a lot something about scan2ifc

I 2nd this, as well.

I like it too and without any deep thoughts i feel it might be more accessible to me than paper2ifc at first glance.

But same question. To train model, we would need training data and opendata of scan3D with corresponding ifc model might be difficult to find... I don't know.

T by theoryshaw on 29 Jan 2024, edited 29 Jan 2024 # +2 votes

But same question. To train model, we would need training data and opendata of scan3D with corresponding ifc model might be difficult to find... I don't know.

Some of the latest project repos https://hub.openingdesign.com/OpeningDesign we modeled up the existing conditions from 3d scans done by an iphone 14, using Polycam app. Might help.

The .glb scans are usually in the following path ...Open\Existing Conditions and Code Research\Existing Conditions\Scans

G by Gorgious on 30 Jan 2024, edited 30 Jan 2024 # +1 votes

One big problem also with 3D scans is the huge disk point clouds takes. And even if you manage to produce a 3D mesh out of it, it is usually millions upon millions of polygons which are very hard to retopologize correctly. I don't know if it can really be used to train current generation ML algorithms. Today I think a realistic processing of 3D scans is segmentation, Florent Poux who is active on the subject on Linkedin shared much ressources on the subject.

I never managed to deep dive into it but it seems promising. However it's still reserved for developers or programmers, as the workflow necessitates building your own coding environment and processing everything yourself. Maybe in the future it will be released to a more general audience, but for now it's still very much experimental (IMHO).

B by bdamay on 30 Jan 2024, edited 30 Jan 2024 #

@Gorgious, i have discovered Dr Poux videos last sunday while i was looking around for hints about my subject. It is indeed very promissing.

I asked him about a workflow using LIDAR data to vectorise low poly meshes of geographic context and he seems interested to do a video on the subject.

Concerning Buildings, his point cloud videos are on my personal todo list, i guess have a lot to learn from that. He seems to have developped techniques that might be used to detect objects in a point cloud scene.

It's really cool .

M by Moult on 30 Jan 2024 # +1 votes

@buovjaga said:

Note that when applying for GSoC, you should never point to a list of issues as the ideas page. Using such a link will quite certainly lead to a rejection.

Yes, usually this is the case, except this is a specially created ideas GSoC list (not a bugtracker, even though it does use the Github issues system) created by the umbrella organisation BRL-CAD that has been part of GSoC for many years now. We don't apply directly, we are underneath the umbrella, so we follow the rules of BRL-CAD.

@bdamay have you considered partnering with a company where they can share their BIM models? (other than the very generous open data license from the OpeningDesign projects)

As for point clouds, given their heaviness I think people would generally get more return on investment on 360 photos assuming it's a job that 360 photos can also do like on-site verification.

G by Gorgious on 31 Jan 2024 # +1 votes

FWIW here's simple manual workflow if you want to mesh a low-poly terrain using cloudcompare.

Load in the point cloud

Segment out the buildings and non-terrain features

Cull most points with the subsampling tool

Compute the normals

Reconstruct the mesh with the poisson algorithm (higher octree means more polygons)

Here's your 3D mesh (don't use it as a topographical reference)