OSArch's own GIT hosting service

T by theoryshaw on 20 Nov 2024, edited 6 Jan 2025 # +16 votes

Dear Steering Committee (@brunopostle, @CadGiru, @dimitar, @Moult, @Jesusbill)

...and anyone else interested in chiming in.

I’d like to gather your thoughts on the potential for OSArch to host its own Git platform. As some of you may know, I currently host a Forgejo instance at hub.openingdesign.com. While this setup serves its purpose, I’m excited about the idea of transitioning this instance under the OSArch banner, provided there’s sufficient interest and support from the community.

This platform could serve several functions for OSArch:

Centralized Hosting for IFC Content Libraries

We could use the Forgejo instance to host and maintain IFC content libraries, making them easy for the community to use and contribute to.
Code Sharing and Collaboration

It could also act as a repository for one-off code snippets and tools shared on the OSArch forums, streamlining collaboration and troubleshooting.
Company Projects

Ideally, companies could host their own projects here—whether public or private—encouraging greater adoption of open-source, and IFC-based workflows in our industry.
Tool Development

With tools like @brunopostle’s IfcMerge, this platform could facilitate focused development efforts. For example, we could collaboratively enhance IfcMerge or other community-driven tools.
IFC Workflow Integration

Over time, we could integrate features tailored to IFC workflows, starting with something as simple as an embedded IFC viewer and gradually moving toward more advanced tools like visual IFC diffing and merging.

Costs and Funding

Currently, hosting the Forgejo instance costs about $75/month. OpeningDesign is willing to cover a portion of this cost. However, for long-term sustainability, we might consider a shared funding model. Companies using the service could contribute, or we could establish an ongoing community funding campaign, similar to this example.

Next Steps

If this idea resonates with you, please show your support by liking or commenting on this post. If there’s enough political interest, I’d be excited to move the current instance under the OSArch banner and have its future evolution directed by the community.

Looking forward to your thoughts!

Best regards,

Ryan Schultz

N by Nigel on 20 Nov 2024 # +1 votes

@theoryshaw, this is a great proposal for sharing and collaboration. I am happy to contribute financially to get this initiative going and with the ongoing fees. I'm sure the number of people here willing to contribute would make any individual cost very low.

B by brunopostle on 20 Nov 2024 # +1 votes

Yes, I think this would be a good service. The focus should be on running live projects as a CDE, but also hosting shared library resources.

What kind of deployment would this be?

Is it a managed system with security updates? The version of forgejo on hub.openingdesign.com is only a couple of months old, who is updating this?

If it is managed, how easy is it to support custom extensions? The web-ifc-viewer on https://gitaec.org/ seems to be reasonably well self-contained, but I haven't investigated closely. Have you looked at copying this extension on to your instance?

T by theoryshaw on 20 Nov 2024, edited 20 Nov 2024 #

What kind of deployment would this be?

Is it a managed system with security updates? The version of forgejo on hub.openingdesign.com is only a couple of months old, who is updating this?

I'm currently working with Michael Jerger of meissa-gmbh.de through a paid consultancy. They've...

setup backup systems,
helped delete a bunch of spam accounts when there was no captcha installed
have updated storage capacity
done periodic upgrades

I'm assuming they would help with the transition to OSArch, if we decide to go in that direction. It would be nice too, if there's anyone from the core community here, that would like to help with maintaining and extending as well.

If it is managed, how easy is it to support custom extensions?

Not sure--defer to others.

The web-ifc-viewer on https://gitaec.org/ seems to be reasonably well self-contained, but I haven't investigated closely. Have you looked at copying this extension on to your instance?

Yeah, that sounds like a great option--defer to others more qualified to make that assessment.

C by CadGiru on 21 Nov 2024, edited 21 Nov 2024 #

Great idea

Would like to see a fork that implemented BCF for issue handling

@brunopostle Had a brief look at gitaec, looks promising.

B by brunopostle on 21 Nov 2024 #

@theoryshaw looks like you found a good hosting solution. @CadGiru a BCF integration would be important, but I have no idea what a good workflow would be.

D by dimitar on 25 Nov 2024 # +1 votes

Hi this is a bit to technical for me, but is it sort of like Speckle, in the sense that osarch can offer hosted solutions for a fee for businesses? If not perhaps, it should be this way, as there are definitely associated costs that come with hosting lots info and lots of syncs and pulls.

T by theoryshaw on 25 Nov 2024 # +5 votes

osarch can offer hosted solutions for a fee for businesses?

Yes, the proposal is for it to be self-sustaining through funding contributions. I'm thinking if there's a surplus in funding, that this funding could go toward development projects that add functionality (like an ifc viewer, visual differ, bcf, etc) to the Forgejo instance.

M by Moult on 28 Nov 2024 # +6 votes

Can I perhaps broaden the scope a little bit and ask about whether anybody is aware of promising initiatives for an open source CDE? (something similar to what ThatOpenCompany was promising in the past, until they changed their mind and nothing came out of it)

Viewing an IFC and versioning model uploads is only one aspect of a CDE for collaboration. There is also document storage with metadata (you need to upload thousands of drawings, spec sheets, reports, etc, and each needs to have metadata like status, discipline, revision, document number, document title, ...), there is issue management (which Forgejo issues can sort of work for this, but needs better referencing of documents and models), mail or transmittals (formal correspondence, RFIs, document submissions...), etc etc.

N by Nigel on 29 Nov 2024 #

@Moult said:

Can I perhaps broaden the scope a little bit and ask about whether anybody is aware of promising initiatives...

Hi Dion, are you saying someone/some entity has proposed this or are you proposing this?

would this be an Aconex-ish type of thing? for design and construction, maybe even, asset and maintenance management

or have I miss understood? which is highly likely ;)

J by JanF on 29 Nov 2024 # +1 votes

@Moult said:

Can I perhaps broaden the scope a little bit and ask about whether anybody is aware of promising initiatives for an open source CDE? (something similar to what ThatOpenCompany was promising in the past, until they changed their mind and nothing came out of it)

Viewing an IFC and versioning model uploads is only one aspect of a CDE for collaboration. There is also document storage with metadata (you need to upload thousands of drawings, spec sheets, reports, etc, and each needs to have metadata like status, discipline..

The openproject+nextcloud combo doesn't seem to be doing custom metadata (but other than that is cool!) and that's by far the most evolved os one. So I doubt there's anything better.

M by Moult on 30 Nov 2024 # +4 votes

I'm just fishing in case someone knows something about the state of open source CDEs that's more than what's been mentioned here. I would love an open source Aconex replacement.

And if someone is aware of something, we should totally encourage it too and host that.

I was quite surprised to hear NextCloud doesn't support custom file metadata, that seems a real shame. To me the most promising approach would be to build something on top of Forgejo, with Git-LFS to store docs, but say SQL db on top to store all the file metadata. It doesn't sound too hard, and potentially something we can fund for a basic webapp.

B by brunopostle on 30 Nov 2024 # +5 votes

In addition to making changes necessary to repurpose the tracker functionality for project communication, there is a whole lot of continuous integration-type stuff that could be done: interrogating every commit or tag and generating Gantt charts; drawing sets; costing schedules and cashflow projections; IDS and IFC validation reports; energy and carbon analysis.

I'm not a fan of 'traffic-light' dashboard overviews, but a lot of this information can be summarised.

The html reporting in the web-gui needs to be detached from blender and made available to this project git forge.

We should be looking at new workflows where multiple disciplines are working on the same model: the tools need to be checking that structural members correspond to related building elements; that element geometry changes don't indicate that qsets need updating; that element weight changes don't imply structural changes (even that every element has a calculable weight); whether there are elements without tasks or costing; that there are elements or properties in the model that don't appear in drawings or schedules; a change in space occupancy should flag potential M&E or structural consideration etc.. etc..

S by sjb007 on 30 Nov 2024 #

@Moult said:

I was quite surprised to hear NextCloud doesn't support custom file metadata, that seems a real shame.

I don't really know anything about the needs and requirements you have a for a viable solution, but when did you last check this?

https://docs.nextcloud.com/server/stable/developer_manual/digging_deeper/files-metadata.html

Maybe I misunderstand that page, but it looks like custom metadata on files, and there is a query builder too so you should be able to search by metadata.

J by JanF on 30 Nov 2024 #

@sjb007 said:

I don't really know anything about the needs and requirements you have a for a viable solution, but when did you last check this?

https://docs.nextcloud.com/server/stable/developer_manual/digging_deeper/files-metadata.html

Maybe I misunderstand that page, but it looks like custom metadata on files, and there is a query builder too so you should be able to search by metadata.

A, I understood it exactly the other way around, that this is only about file metadata

B, this is only the API, it doesn't mean it's implemented for the users in any meaningful way

I have only found this discussion which seems to be a dead end:

https://help.nextcloud.com/t/files-extended-metadata/2487

I'd be happy if proven wron, because I was planning on switching to nextcloud from gdrive and would really like to have custom metadata.

S by sjb007 on 30 Nov 2024 #

@JanF said:

@sjb007 said:

I don't really know anything about the needs and requirements you have a for a viable solution, but when did you last check this?

https://docs.nextcloud.com/server/stable/developer_manual/digging_deeper/files-metadata.html

Maybe I misunderstand that page, but it looks like custom metadata on files, and there is a query builder too so you should be able to search by metadata.

A, I understood it exactly the other way around, that this is only about file metadata

B, this is only the API, it doesn't mean it's implemented for the users in any meaningful way

I have only found this discussion which seems to be a dead end:

https://help.nextcloud.com/t/files-extended-metadata/2487

I'd be happy if proven wron, because I was planning on switching to nextcloud from gdrive and would really like to have custom metadata.

Well, v28 (when it was added) was released a little under a year ago, and the feature doesn't seem to have much fanfare - there's nothing in the release announcement. Maybe people just haven't noticed... :-)

A. From the examples:

$metadataManager->initMetadata('myapp-test', IMetadataValueWrapper::TYPE_INT, true, IMetadataValueWrapper::EDIT_REQ_OWNERSHIP);

<nc:metadata-myapp-test>123</nc:metadata-myapp-test>

This implies application specific, self named and typed key/value pairs per file. To me, it doesn't read as limited to filesystem metadata. Or am I misunderstanding? Are you referring to having metadata on non-file items?

B. It has WebDAV, so can be set, read and queried over HTTP queries, from Bonsai for example.

You might need a small app registered so that the myapp part of the nc:metadata-myapp-test is valid, and a paragraph on the page leads me to believe that the key/types themselves would need to be defined in there too. But maybe here you are talking about a NextCloud UI for accessing and editing the metadata? OK, sure, so maybe that needs to be added, but it seems to me that adding that small functionality to NextCloud is a lot less work than building a Frankenstein CDE/DMS out of disparate software. NextCloud already seems to have lots of the pieces, so why reinvent them?

J by Jesusbill on 3 Dec 2024 #

Hi all, I support the proposal of creating an OSArch GIT hosting service as proposed by @theoryshaw, and that this should be part of a greater plan of promoting NativeIFC integration of workflows among disciplines and such.

I believe we need to have a clear idea of the costs associated with the service; I'd say:

Setup and maintenance cost (recurring) for any external DevOps needed
Cost of server instance with an approx. projection on number of users
Cost of storage with an approx. projection on number of users
Other ...

Since OSArch is not a company and cannot hold responsibility, I imagine it would be better to have a donation-based service and not a professional service, and try to collect funds for this purpose from interested companies and individuals.

I would not underestimate the complexity of launching such a project, though, and various misuses that can happen. Allowing unlimited projects for example without any control, even from a paid member, could become a problem. So, we should assess all these risks and proceed with care.

@theoryshaw I would be happy to help you with this and take some role on the DevOps side if no-one more qualified can be found. I have done quite some work on full-stack developments lately, using DevOps services though like vercel and render. This is an alternative way of deploying as a service, more flexible but probably not applicable for this case, if the Forgejo instance works as a monolith.

T by theoryshaw on 4 Dec 2024, edited 4 Dec 2024 # +3 votes

Very cool. I'm excited about this.

I think the first step should be as hyper simple as possible.

point the https://hub.openingdesign.com/ to something like https://hub.osarch.org/ (or if anyone has any ideas other than the prefix 'hub'.)
- @dimitar we're going to need your help, creating the subdomain and pointing it to the IP address.
setup a redirect so old links from https://hub.openingdesign.com/ still point to the correct corresponding location on hub.osarch.org/
update the instance with OSArch logos, etc.
Keep the manual way to register for now, but provide OSArch admin email.
- We decided to go with this manual way to register because the default captcha was not working, as there was a lot of spamming accounts.
Give SSH and AWS admin access to steering committee members. @Moult, @brunopostle, @Jesusbill
- @dimitar and @CadGiru, did you want access as well?
Yes, @Jesusbill that would cool with me if you could help with the DevOps! Along with @Moult and @brunopostle, if they're willing. :)

Let me know if anyone has any additional ideas for this first step... but please keep it MVP. :)

I have pinged, meissa GmbH, to this conversation. I think it makes sense for them to do these initial steps. Once we settle on these first steps, i'll have them put together an estimate.

Looking forward to the evolution!

Best, Ryan

M by Moult on 4 Dec 2024 #

The DNS is provided by Cloudflare, so when we know where it's hosted I can help tweak the settings.

N by Nigel on 4 Dec 2024 # +1 votes

It would be good to provide an elevator pitch so I and maybe others understand what is being proposed

S by steverugi on 4 Dec 2024 #

@Nigel

It would be good to provide an elevator pitch so I and maybe others understand what is being proposed

I too need an elevator pitch :)

C by CadGiru on 4 Dec 2024 #

@theoryshaw pls Count me in, great initiative

B by brunopostle on 4 Dec 2024 # +5 votes

@Nigel said:

It would be good to provide an elevator pitch so I and maybe others understand what is being proposed

It is possible that we don't have a clear idea of what this is.

An AEC git-forge seems like an obvious next step, we can do design collaboration with IFC just like we do with code. The reason for doing this is that with better team integration, and automation tricks learnt from software development, we can design better buildings.

The issue tracker in a git-forge looks a lot like a CDE: there is communication between team members, issues can be delegated, tracked and resolved etc.. but CDEs are a particular thing, they are primarily file-sharing sites for all the formal legally-required communication involved in delivering construction projects. In my experience, CDEs are full of PDFs, occasionally snapshots of revit, dwg or microstation files (I've never seen an iFC file, but maybe this is just the projects I see). Often these PDFs are letters, printed, signed and scanned - everything is a legal contractual document with an eye to eventual litigation.

Now git is a terrible medium for sharing PDFs and proprietary file formats, even SVGs are a problem in git, a file-sharing service like nextcloud seems like a much better fit, but how or why this interfaces with a git forge I don't really know. Certainly any CI process that publishes drawing sets from tags in an IFC model should publish those drawings to something like nextcloud rather than adding them to a git repository. BCF, since it is so closely tied to IFC could be integrated nicely with a git-forge issue tracker that has a built-in IFC viewer.

IFC and Bonsai has great pricing and scheduling built in, I can see that an all-in-one design-build organisation could use a git-forge for everything from architecture and engineering, to project management and construction - this could be a whole new way of working. But where a git-forge doesn't fit nicely is in a project where professionals, suppliers, contractors and clients are effectively in opposition.

J by Jesusbill on 4 Dec 2024 # +2 votes

I think the "pitch" should focus on the benefits of using NativeIFC tools (open data, low-costs, rich schema across disciplines, ...) and how this hub can help face the challenges of transitioning to NativeIFC through shared data (IFC Content Libraries, Projects, etc) and through collaboration with qualified users. I believe NativeIFC should be the "moto"

J by Jesusbill on 4 Dec 2024 # +4 votes

It means no worries for the rest of your days ♫ ♪ ♬

It's our problem-free philosophy, hakuna matata ♫ ♪ ♬

♫ ♪ ♬