Hi Leif,
I added 2 pipes to buildin.py:
- publish_html creates static HTML views of IDPs and SPs, using XSLT based on Peter Schober’s alternative to MET;
- publish_split: similar to store, but added validUntil and creates signed XML-file per EntityDescriptor. This can be consumed dynamically by ADFS in an IDP role.
I put it directly into buildin.py because it shares some code with the sign pipe. Is this viable from your PoV - if yes, I would make an PR.
Cheers, Rainer
Hi all,
being part of Commons Conservancy brought up yet another subject,
which is whether we should add a header with license information in
every file in the projects under idpy. This is not something done in
an abstract way, there is a specific format modelling this information
(see https://spdx.org/ and https://reuse.software/ - more specifically
https://reuse.software/practices/2.0/) Still, I find it problematic.
We want to open up the question to the wider community and consider
their thoughts on this. The forwarded message below is discussing this
subject. You can see the question we posed, the answer we got and my
comments. Feel free to tell us what you think on this.
---------- Forwarded message ---------
Date: Thu, 16 May 2019 at 09:56
> ---------- Forwarded message ----------
> Date: May 8, 2019, 8:15 AM -0700
>
> > Why does CC think having a single license file per project is
> > insufficient? Our thought is that if we can avoid adding a header to
> > every single file, that would be nice, esp. given we already have this
> > info in the license file and we have the Note Well.
>
>
> this is not just our opinion, but something that is an industry and
> community standard for legal compliance these days. When companies like
> Siemens, Samsung or Honeywell use some code in one of the hundreds or
> thousands of devices and systems in their product line, they need to be
> able to provide the correct license and a download of the exact version.
> This means machine readability too.
>
I've actually observed the opposite of that. Communities abandon the
"license in every file" model, and just use a single LICENSE file in
the root of the project. The LICENSE file contains license
information, that is, it is not a single license but it has exception
sections and so on.
> To quote from https://reuse.software/practices/2.0/ :
>
> Scroll to the section "2. Include a copyright notice and license in each
> file"...
>
> "Source code files are often reused across multiple projects, taken from
> their origin and repurposed, or otherwise end up in repositories where
> they are separate from its origin. You should therefore ensure that all
> files in your project have a comment header that convey that file’s
> copyright and license information: Who are the copyright holders and
> under which license(s) do they release the file?
>
Continuing from above, the standardization of package-management
formats and tools has helped exactly with that: to avoid distribution
of single files, and instead provide packages and modules. It is bad
practice and considered a hack to copy files. Nobody liked that model
and everyone is moving away; it is unstructured, it becomes
unmanageable and it will cause problems.
> It is highly recommended that you keep the format of these headers
> consistent across your files. It is important, however, that you do not
> remove any information from headers in files of which you are not the
> sole author.
>
> You must convey the license information of your source code file in a
> standardised way, so that computers can interpret it. You can do this
> with an SPDX-License-Identifier tag followed by an SPDX expression
> defined by the SPDX specifications."
>
> (the text goes on for a while after this, to clarify the point but this
> is the basic gist of it)
>
> There is a nice Python tool to check:
>
> https://github.com/fsfe/reuse-tool
>
> I hope this makes sense
>
Well, it does not make complete sense. We're talking about licensing a
project. A project is not just code; there are data files (html, xml,
yaml, json files), binary files (archives/zip, images, audio, video,
etc), text files (configs, ini-files, etc) all "not-code". How do you
mark those files? Does the LICENSE file need a license-header? The
json format does not define comments, how do you add a header there?
If a binary file does not get a license header, why should a file with
code get one?
I would expect there to be a way to have the needed information
unified. If the files themselves cannot provide this information it
has to be external; thus the LICENSE file. If someone is worried about
somebody else re-using single files that do not have license
information (a python file, a png image, etc) there is really nothing
you can do (the DRM industry has been trying to solve for a long time;
and still your best bet is "social DRM").
Since, we're developing on open source with a permissive license, even
if someone does that, should we be happy that someone is actually
using what we built or sad that the files they copied did not have a
license header? And if they include the license information of that
copied file in their project's LICENSE file, is this solved?
Having pointed these contradictions, I am thinking that the "license
in every file" model seems to be a step backwards. It is introducing
overhead and does not really solve the problem, while at the same time
it enables a culture of bad practice (copying files around).
Cheers,
--
Ivan c00kiemon5ter Kanakarakis >:3
Attendees:
Roland, Johan, Heather, Scott, Ivan, Matthew
Notes:
0 - Agenda bash
1 - Administrivia
• Board will have their annual meeting next week. Same board slate.
• Johan will be off for parental leave until August 2023
2 - Project review
a. General
b. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
Roland has been working on documentation aimed towards software developers.
Working on the PRs in the OIDC libraries and have merged all but one. Also started rebasing the fedservice fork so we can issue a PR soon for the work done to support federation. This will be a major PR but mostly in the lower level code. This will bring eduTEAMS in sync with this library; at that point there will be no reason to keep separate software running for eduTEAMS.
OIDC federation spec is getting closer to being finalized. There have been discussion from people running OIDC federations in the same way that SAML federations are run (one authority collecting the info). There is a difficulty in that one model (OIDC federation spec) assumes end-to-end encryption where the other assumes that encryption stops at the edge and the organization can inspect the payload (required by regulation for banking). Finding a compromise by allowing metadata collection from well-known endpoints. This involves separating the path from the trust model.
c. Satosa - https://github.com/IdentityPython/SATOSA
Ivan is preparing a new release, but nothing major. Is considering accepting one more PR (https://github.com/IdentityPython/SATOSA/pull/429) and then will cut the new release. See also
• https://github.com/IdentityPython/SATOSA/issues/428
• https://github.com/IdentityPython/SATOSA/pull/430
After this release, will move Satosa to using poetry and require Python 3.9.
d. pySAML2 - https://github.com/IdentityPython/pysaml2
Next release will see minimum Python requirement to 3.9 (see email to list).
Other major updates:
• https://github.com/IdentityPython/pysaml2/pull/888 (may need further discussion)
• https://github.com/IdentityPython/pysaml2/pull/895 (compatibility changes)
A user came back to an issue (submitted as a PR: https://github.com/IdentityPython/pysaml2/pull/665) about how operations are done with xmlsec1. May need to revisit this. By default, pySAML2 uses the xmlsec1 binary. xmlsec1 works with files, so pySAML2 is always writing files to the file system, but Windows cannot automatically clean up those files; they have to have an external process cleaning up the files. This is not fixable. for the *nix systems, there are automatic cleanups, but they are buggy. Person who submitted the issue offered a suggestion (code) but another option is to handle xmlsec1 differently and offer a different back end. If interested, please review and comment on the PR.
e. Any other project (pyFF, djangosaml2, etc)
3 - Documentation
See OIDC update
4 - AOB
Thanks! Heather
Hello everyone,
Within IdentityPython we have agreed to support the Python version
that comes with the stable version of Debian. The current stable
distribution of Debian is version 11, codenamed bullseye. It was
initially released as version 11.0 on August 14th, 2021 and its latest
update, version 11.6, was released on December 17th, 2022.
Debian stable supports Python 3.9, so I am looking towards bumping the
minimum supported Python version to 3.9, for PySAML2 and then for
SaToSa. Once this is done, we will be adding typing information to the
codebases to ease contributions from other developers and start
separating the public APIs from internal code. This will aid in
documenting what is there, refactoring the code and cleaning up old
and unused code.
The plan is to create a new release with the currently merged changes.
Then, have a separate release just bumping the minimum Python version
required by the package.
Expect these new releases to happen within the next week.
Cheers,
--
Ivan Kanakarakis - sunet.se
Attendees:
Matthew, Roland, Johan, Ivan, Heather, Giuseppe
Note:
0 - Agenda bash
• py11 - who is maintaining this?
1 - Administrivia
a. Board slate for 2023
• Heather Flanagan (Spherical Cow Group)
• Leif Johansson (SUNET)
• Christos Kanellopoulos (GÉANT)
2 - Project review
a. General
• python minimal version update; first with pySAML2 (see notes below) and later to Satosa and other libraries
b. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
• Roland's video is now on YouTube. While creating the session, found things to change in the code to be more consistent. That's been complete and the OIDC federation implementation has been updated as well. Question now is what to do with the older PRs from eduTEAMS. Should those PRs be applied first, then Roland's changes, or vice versa? When the work is complete, then Roland will not do any new development (bug fixes only) unless new functionality is absolutely required. Instead will focus on writing documentation for other core developers (not users)
• PR for client credentials - nice to have, not necessary
• PR for revocation endpoint - that is deployed and running in eduTEAMS; it should come with tests and ok to merge
• PR for various changes - fixing or changing some things that came when eduTEAMS migrated to idpy oidc; changes that were never merged on oidc op.
c. Satosa - https://github.com/IdentityPython/SATOSA
One new issue reported - desire for support for more client authentication methods. Don't want to do this on pyop; should try to push them to use the new front ends.
Still wants to move Satosa to using poetry, but hasn't had time yet.
d. pySAML2 - https://github.com/IdentityPython/pysaml2
• bumping the minimum supported python version to 3.9 (as supported by latest stable Debian) - some of the updates to various libraries would be much nicer if we stick with a more recent version of python; would be able to use types (among other things). Primary cause for using types is support for better documentation. There are tools that will allow types to be stripped from the code if people in older environments need that. idpy 3.7 is the oldest supported python release, but support for that ends in June.
• Ivan to send a note to the mailing list letting people know about this change. Bumping the python version will be its own release for pySAML2, then another release adding support for mypy, then a separate release adding support for types.
• start adding typing support (and use mypy) - see python 3.9 discussion. Once you have enough info through the types, it is easier to refactor the code because you know what's there, what cases need to be covered.
• this is done in the context of Johan and Fredrik working on adding support for the Anonymous/Pseudonymous/Personalized entity categories
e. Any other project (pyFF, djangosaml2, etc)
djangosaml2 - new release out; there was a regression from the previous release.
pyFF - Matthew is planning to develop a docker official image for pyFF sometime this spring, similar to what he's developed for Satosa. What's on dockerhub now is old and not obviously maintained. Nicole Roy will also be part of this work.
• eduTEAMS uses pyFF; they have forked the image and updated it with minor modifications. The problem with pyFF is that the bigger the metadata you give it, the more memory it requires. Could look into breaking the operation into chunks. The only reason the whole payload is required is to calculate the final signature, but that can be done incrementally. Unclear if this change would break anything. eduTEAMS (Ivan) has investigating this on the list, but it's not a high priority. This would be an internal change.
• SUNET has also requested some changes and have created a branch with improvements. Need to ask Leif about this.
py11 - ok for Johan to be the maintainer and merge his PR? Yes.
3 - Documentation
Doc writing guidance:
• https://www.writethedocs.org/videos/eu/2017/the-four-kinds-of-documentation…
• https://developers.google.com/tech-writing
• https://www.technical-communication.org/technical-communication/software-do…
• https://ieeexplore.ieee.org/document/625327 (pay walled)
Should we post a survey to collect information on the developer's experience to help prioritize/guide our documentation? When a developer lands on the idpy pages, do they have enough information to start contributing and engaging with the community? There are different types of developers - developers who are trying to deploy it, developers who want to add functionality, developers who are trying to find an answer to a specific question.
• Roland is focusing on writing documentation for developers who have a specific problem with the current OIDC libraries
• Heather and Ivan to review https://github.com/IdentityPython/Governance/blob/master/idpy-projects.md to see if it needs to be updated (see section on Technical quality)
Thanks! Heather
Tuesday, 17 January 2023, 14:00 UTC
Zoom: https://us06web.zoom.us/j/83378219417?pwd=dWFLdjRHK3BnRkZMa3VSd2lNaElpdz09
Agenda:
0 - Agenda bash
1 - Administrivia
2 - Project review
a. General
• bumping the minimum supported python version to 3.9 (as supported by latest stable Debian)
• start adding typing support (and use mypy)
• this is done in the context of Johan and Fredrik working on adding support for the Anonymous/Pseudonymous/Personalized entity categories
b. OIDC - https://github.com/IdentityPython (JWTConnect-Python-OidcRP, JWTConnect-Python-CryptoJWT, etc)
c. Satosa - https://github.com/IdentityPython/SATOSA
d. pySAML2 - https://github.com/IdentityPython/pysaml2
e. Any other project (pyFF, djangosaml2, etc)
3 - Documentation
4 - AOB
Thanks! Heather
Hello idpy developers!
It’s that time again! Time to accept nominations for the Board slate. Three of the current board members are at the end of their term; all three have agreed to continue on as Board members:
• Heather Flanagan (Spherical Cow Group)
• Leif Johansson (SUNET)
• Christos Kanellopoulos (GÉANT)
As a reminder regarding board seats:
"The IdentityPython Board has a minimum of three, and a maximum of seven natural persons. The founding IdentityPython Board has appointed a number of its constituting Directors to serve a half (12 month) term, and the remainder to serve a regular (24 month) term. Subsequent Directors are elected by the IdentityPython Board to regular 24-month terms according to the procedure set out in these Statutes. The founding Board will select a nominating committee of active developers and other contributors to identify candidates for ongoing Board membership. Directors are permitted to seek office for multiple terms, however, when running against other candidates the amount of terms they have consecutively served is deducted from the votes cast in their favour. This provides a balance between continuity, equal opportunities and renewal of qualities and competences."
(From the Identity Python statutes, officially published with The Commons Conservancy: https://dracc.commonsconservancy.org/0024/)
If you have interest in serving on the board, please let Ivan or I know. Nominations are open until 15 January 2023. If no new nominations come in, the board will re-appoint the current members.
Thanks! Heather
Hello all!
A new year means a new calendar series for the idpy developers calls! For several of the regular attendees, I’ve added you directly to the calendar invitation. For others, if you would prefer to add the call directly to your calendar, you should be able to use this link to add the meeting.
Our next call will be next Tuesday, 17 January 2023, at 14:00 UTC (it’s pinned to 6am US Pacific time).
Please let me know if you have any questions, and I will see you next week!
Thanks! Heather
