Hi Leif,
I added 2 pipes to buildin.py:
- publish_html creates static HTML views of IDPs and SPs, using XSLT based on Peter Schober’s alternative to MET;
- publish_split: similar to store, but added validUntil and creates signed XML-file per EntityDescriptor. This can be consumed dynamically by ADFS in an IDP role.
I put it directly into buildin.py because it shares some code with the sign pipe. Is this viable from your PoV - if yes, I would make an PR.
Cheers, Rainer
Hi all,
being part of Commons Conservancy brought up yet another subject,
which is whether we should add a header with license information in
every file in the projects under idpy. This is not something done in
an abstract way, there is a specific format modelling this information
(see https://spdx.org/ and https://reuse.software/ - more specifically
https://reuse.software/practices/2.0/) Still, I find it problematic.
We want to open up the question to the wider community and consider
their thoughts on this. The forwarded message below is discussing this
subject. You can see the question we posed, the answer we got and my
comments. Feel free to tell us what you think on this.
---------- Forwarded message ---------
Date: Thu, 16 May 2019 at 09:56
> ---------- Forwarded message ----------
> Date: May 8, 2019, 8:15 AM -0700
>
> > Why does CC think having a single license file per project is
> > insufficient? Our thought is that if we can avoid adding a header to
> > every single file, that would be nice, esp. given we already have this
> > info in the license file and we have the Note Well.
>
>
> this is not just our opinion, but something that is an industry and
> community standard for legal compliance these days. When companies like
> Siemens, Samsung or Honeywell use some code in one of the hundreds or
> thousands of devices and systems in their product line, they need to be
> able to provide the correct license and a download of the exact version.
> This means machine readability too.
>
I've actually observed the opposite of that. Communities abandon the
"license in every file" model, and just use a single LICENSE file in
the root of the project. The LICENSE file contains license
information, that is, it is not a single license but it has exception
sections and so on.
> To quote from https://reuse.software/practices/2.0/ :
>
> Scroll to the section "2. Include a copyright notice and license in each
> file"...
>
> "Source code files are often reused across multiple projects, taken from
> their origin and repurposed, or otherwise end up in repositories where
> they are separate from its origin. You should therefore ensure that all
> files in your project have a comment header that convey that file’s
> copyright and license information: Who are the copyright holders and
> under which license(s) do they release the file?
>
Continuing from above, the standardization of package-management
formats and tools has helped exactly with that: to avoid distribution
of single files, and instead provide packages and modules. It is bad
practice and considered a hack to copy files. Nobody liked that model
and everyone is moving away; it is unstructured, it becomes
unmanageable and it will cause problems.
> It is highly recommended that you keep the format of these headers
> consistent across your files. It is important, however, that you do not
> remove any information from headers in files of which you are not the
> sole author.
>
> You must convey the license information of your source code file in a
> standardised way, so that computers can interpret it. You can do this
> with an SPDX-License-Identifier tag followed by an SPDX expression
> defined by the SPDX specifications."
>
> (the text goes on for a while after this, to clarify the point but this
> is the basic gist of it)
>
> There is a nice Python tool to check:
>
> https://github.com/fsfe/reuse-tool
>
> I hope this makes sense
>
Well, it does not make complete sense. We're talking about licensing a
project. A project is not just code; there are data files (html, xml,
yaml, json files), binary files (archives/zip, images, audio, video,
etc), text files (configs, ini-files, etc) all "not-code". How do you
mark those files? Does the LICENSE file need a license-header? The
json format does not define comments, how do you add a header there?
If a binary file does not get a license header, why should a file with
code get one?
I would expect there to be a way to have the needed information
unified. If the files themselves cannot provide this information it
has to be external; thus the LICENSE file. If someone is worried about
somebody else re-using single files that do not have license
information (a python file, a png image, etc) there is really nothing
you can do (the DRM industry has been trying to solve for a long time;
and still your best bet is "social DRM").
Since, we're developing on open source with a permissive license, even
if someone does that, should we be happy that someone is actually
using what we built or sad that the files they copied did not have a
license header? And if they include the license information of that
copied file in their project's LICENSE file, is this solved?
Having pointed these contradictions, I am thinking that the "license
in every file" model seems to be a step backwards. It is introducing
overhead and does not really solve the problem, while at the same time
it enables a culture of bad practice (copying files around).
Cheers,
--
Ivan c00kiemon5ter Kanakarakis >:3
*Idpy meeting 26 September 2023*
Attendees: Ivan, Johan, Matthew, Hannah, Shayna
0 - Agenda bash
1 - Project review
a. General -
- Kushal Das - new to idpy - will initialyl focus on configurations
b. OIDC libraries - https://github.com/IdentityPython (idpy-oidc,
JWTConnect-Python-CryptoJWT, etc)
- Matthew is deploying satosa-oidcop. What are the plans for integrating
this with a new release of SATOSA? The old oidc-op does not work out of the
box with SATOSA 8.4 and the current version pyop. Since this is kind of a
breaking change, his recommendation is to go ahead and make the jump to the
new satosa-oidcop. Enterprise deployments don't work with old version of
oidc-op since pymongo has moved on. Matthew has concerns over the docker
container - it should be release quality for enterprise users. He would be
happy to bake satosa-oidcop into the next version of the container. Or
should he stay in lock step with SATOSA releases?
- Ivan says they can make changes to make pyop work and release. Will
leave it in since some people are using it.
- The plan is to keep everything and allow people to use frontends by
including them and configuring them on the deployment side.
- the plan is not to include satosa-oidcop in the SATOSA repo. It has
its own repo with its own lifecycle and dependencies. They will
keep things
aligned, though, since people will be using it and will report
if something
breaks.
- satosa-oidcop could be brought into idpy but it is up to Giuseppe.
- There is another frontend that should have been released already by
GÉANT but Christos was at TechEx and has had some obstacles. That would
also be a separate repository with its own release schedule.
- Ivan says it is ok to incorporate multiple frontends into the
container and allow the user to configure whichever ones they want.
- SATOSA is intended to be a building block - the docker image is the
integrator so that you can actually use the product and bring it
up. SATOSA
is the core.
- Roland's PR to separate SAML parts from SATOSA is evidence of this
separation. We shouldn't have to build with pysaml2 in SATOSA if
you don't
need it. You could even use another SAML library. SATOSA is a
plugin system
that gives you a core internal abstraction of information for
authentication and authorization which is then translated to
multiple other
protocols, and you can use whatever libraries you want to use.
- Matthew still has a PR on pyop.
- Ivan has questions on how to use idpy-oidc to configure a client that
wants to use the client secret jwt authentication method - will post to see
if Giuseppe or Roland can answer
c. Satosa - https://github.com/IdentityPython/SATOSA
- Matthew is sending slides to the mailing list for single logout that
were presented at TechEx. Meeting will be 10/4.
http://internet2.edu/wp-content/uploads/2023/09/20230920-sebuliba-satosa-sl…
.
- A new SATOSA release is imminent. There are few more PRs to merge:
- - https://github.com/IdentityPython/SATOSA/pull/427 - SATOSA maps
between protocols using configuration that lists what should happen with
particular profiles. Apple does things differently - it returns
attributes/claims that are dictionaries with their own keys and values.
This causes conflicts when trying to look up mappings. The choice is to
either make the code more flexible, not throw exceptions and just ignore
those keys/mappings, or separate Apple to it own profile, which
would make
things things more verbose but also more explicit. Since things are only
slightly different with Apple signing, Ivan's first instinct is to merge
with the oidc backend and make code more flexible. But if Apple changes
more things (which is likely) , we should probably be treating it as
something different.
- - https://github.com/IdentityPython/SATOSA/pull/419- has to do with
how we map some concepts between saml and oidc. It is mapping saml's
is_passive to another concept within oidc, which is the ability
to request
offline access to the user's information even when the user is
not present,
which oidc recommends using a prompt for the user to approve. This MR is
about making is_passive a hint to disable the prompt. (prompt = no). Oidc
spec allows this but strongly recommends using the prompt. We should make
this something that is explicitly configured - available but disabled by
default. Needs some work - Code changes and documenting the
option and its
consequences clearly.
- https://github.com/IdentityPython/SATOSA/pull/441 - SATOSA yaml
parser needs to be extended. Already merged.
- https://github.com/IdentityPython/SATOSA/pull/442 - Roland's work.
There is some work to be done with correctly configuring the
dependencies,
and adding error messages when SAML is not there - alert user
that if they
want to use saml, they need to take special action now. This is
a breaking
change so will probably not be merged right now, it will be in its own
release - with info how to get previously expected behavior.
- https://github.com/IdentityPython/SATOSA/pull/435 - still thinking
on this one - once it is done, people will expand and use the
typing - may
bring in Kushal on this one. Open to suggestions.
d. pySAML2 - https://github.com/IdentityPython/pysaml2
- Not much work done recently - minor updates on dependencies - merged a
PR about clearing up log messages.
- https://github.com/IdentityPython/pysaml2/issues/917 - issue with
importlib-sources. -Python support provided the ability to manage
files/folders (resources) - you could ask the package for info on resource
and didn't need to know the specific path. In the next release they
deprecated some functions that were being used. Now things are breaking for
all Python versions - need to figure out how to do things in a different
way.
- Matthew E wrote something similar for a personal project- uses
packageutil and importlib. Matthew will send to Ivan and Ivan will take a
look to see if it's helpful.
- Python packages are zip files - there are ways to install without
unzipping. So sometimes you are not looking at actual files and
directories, but instead you are looking at a virtual unzipped image in
memory. At that point the path is a virtual thing - a memory
pointer. Ivan
needs to figure out how to reach into these virtual places using the
methods that are left.
- Ivan will look into open issues and then prepare a release for
pysaml2
e. Any other project (pyFF, djangosaml2, pyMDOC-CBOR, etc)
- a new release of pyXMLSecurity came out August 24 - semantic
versioning was added. One PR had to do with self signed or CA certificates,
whether the signature is correct. pysaml2 backend can also use
pyxmlsecurity - Ivan needs to check that it still works.
- waiting for new release for pyff
- FED CM - there are recent developments in the w3c privacy group and
the federated identity community group (previously the federated credential
manager community group) - chrome has a new API they are proposing called
FedCM - to work around google identity access management library that
relies on third party cookies. They want to kill link decorations as well -
they'll break saml, oauth2, oidc, etc. When an IdP needs to do something,
or a RP needs to send a user to an IdP, FedCM can instruct the browser
using this javascript API to get consent from the user to allow the link
decoration and third party cookies before going to the next "hop". For
authentication intermediaries, there are a lot of implications with this.
People involved in talks about this: Zacharias Törnblom (Sunet), Philip
Smart at Jisc (has a demo) , Chris Phillips (Canarie), Leif, Albert Wu,
Nicole Roy, Scott Cantor. See:
https://wiki.refeds.org/display/GROUPS/Browser+Changes+and+Federation
- This will be a particular problem for Research Collaborations using
the AARC blueprint
- These escalation prompts may hide discovery and drive idp toward
Google
- Google will turn this on progressively, possibly starting in
November.
- Invite Judith Bush to talk ?
- questions on whether this will actually move forward, who will
follow suit- Chrome and Google is what we're talking about here
-Mozilla &
Microsoft are more into those discussions. Apple/Safari seems to be doing
its own thing. Vittorio Bertocci was also in the discussion but
has dropped
due to illness. The companies he was involved in, Okta and Auth0 , are
against the changes - also probably the banks . Google wants to
enable but
keeps postponing - saying 2024.
2 - AOB
- Next meeting 10 October 2023
Thanks,
Shayna
There will be a review of PR-431
<https://github.com/IdentityPython/SATOSA/pull/431> on 4 October 2023 from
13:00 to 14:00 UTC.
Please let me know if you would like to attend and I will add you to the
meeting invitation.
Thanks you,
Shayna Atkinson
satkinson(a)sphericalcowgroup.com
Idpy meeting 12 September 2023
Attendees: Giuseppe, Ivan, Johan, Matthew, Hannah, Shayna
0 - Agenda bash
1 - Project review
a. General
b. OIDC libraries - https://github.com/IdentityPython (idpy-oidc,
JWTConnect-Python-CryptoJWT, etc)
- Review of Matthew E's pull request
<https://github.com/IdentityPython/pyop/pull/52> on pyop and
continuation of discussion that was started in Slack.
- Summary : pyop (somewhat deprecated) is not compatible with PyMongo
version 4+. Yet SATOSA depends on pyop for the OIDC front end.
Pull request
is a one line pin to the old version of PyMongo.
- If the new front end is the way of the future, there should be some
documentation that indicates that pyop should not be chosen for new
deployments of OIDC.
- Ivan could not find an example of a client connecting to the
frontend; he should probably provide some examples of static and dynamic
client registration.
- Ivan says pyop is not deprecated but is also not properly
maintained. Most of the effort is being put into idpy-oidc. However, pyop
can still work with restrictions on dependencies. At some point pyop will
be archived and everything will be switched to idpy-oidc. When this
happens, they will not remove the existing packages/frontends that are
available on SATOSA. Instead, deployers can choose to use either
the old or
new frontends/backends. So pyop is not replaced; the deployer
chooses which
ones to use. Pyop code base will no longer be supported by the team, but
developers can make their own forks and make changes as they wish.
Deployers will be pushed toward idpy-oidc. however, since this
is where the
focus will be. This would appear to make it so there is not a breaking
release; however there will be one just due to the special
syntax required
to declare the dependencies you want. For instance you usually say:
pip install satosa
but then you will have to say something like:
pip install satosa[idpy_oidc_backend]
or
pip install satosa[idpy_oidc_backend,saml2_frontend]
etc.
This is already done for the new backend - example in
https://github.com/IdentityPython/SATOSA/blob/628ee94/setup.py#L34 line 34.
If you specify idpy_oidc_backend then this pulls in at least version 2.1.0
of idpyoidc. There will be similar things for saml, etc. and it is already
available for choosing mongo or redis:
"pyop_mongo": ["pyop[mongo]"],
"pyop_redis": ["pyop[redis]"]
- This does have implications on the container/packaging side. The goal
is to be more inclusive, but people can always create their own
containers.
- This document talks about other repos relevant to SATOSA -
https://github.com/IdentityPython/SATOSA/blob/master/doc/README.md#external…
One is satosa-oidcop.
- Roland has requested to change how work is done in the idpy-oidc repo.
Previously there were both main and develop branches, and all work was done
locally against the develop branch. Eventually the develop branch would be
merged back to the main branch and there would be a new release. Roland
proposes working on the main branch directly. Releases are tagged as
necessary.
- Christos and Roland are going to make a release of the OIDC frontend
that is used by eduTeams, probably this week.
- Note: satosa-oidcop is working with MongoDB/pyMongo; the one used by
GEANT eduTeams uses postgres. Doing this with something like Alembic might
be better in the future. Some of the reason for using Mongo was based on
that Roland initially coded using the filesystem for the storage layer, not
a relational approach. Using a relational database requires rebuilding all
the relationships, do cleanup, etc. Giuseppe used the load() and dump()
abstract interface to the internal session storage engine. You can see
practical examples of when these methods are used in satosa-oidcop.
c. Satosa - https://github.com/IdentityPython/SATOSA
- Roland's PR for removing saml dependencies:
https://github.com/IdentityPython/SATOSA/pull/442
- Need to thoroughly review and make sure nothing else breaks. Also
includes replacing some cryptographic operations with cryptography and
cryptojwt libraries.
- Ivan is preparing a new release
- Looking at these PRs
https://github.com/IdentityPython/SATOSA/pull/435 - typing
https://github.com/IdentityPython/SATOSA/pull/438 - saml modules metadata
store
https://github.com/IdentityPython/SATOSA/pull/431 - SLO
- Should the state storage be part of a specific module like saml
backend, or part of core so that any module can use it?
- Requires proper storage that can outlive a reset of SATOSA
- Could be Redis, could be something else, abstract? Redis has
a nice API and Graph database, but there are legal aspects
to its licenses
- Giuseppe shared an abstract storage layer example :
https://github.com/italia/eudi-wallet-it-python/blob/dev/pyeudiw/storage/db….
See line 18 - can use multiple storage engines.
- Giuseppe also shared
https://github.com/italia/eudi-wallet-it-python/tree/dev/pyeudiw
- uses MongoDB by default
- abstraction allows things to be "future-proof", but
abstraction also sometimes prevents being able to use
special features of a
particular solution
- Will also look at how to further separate the core: redoing some of
the login work, enhancing the internals
d. pySAML2 - https://github.com/IdentityPython/pysaml2
- upcoming release , then will continue with other open PRs
- Nice work done regarding having configurable encryption algorithms:
https://github.com/IdentityPython/pysaml2/pull/924
e. Any other project (pyFF, djangosaml2, pyMDOC-CBOR, etc)
- djangosaml2 is stable; Giuseppe is waiting for revisions to be done
that he asked for from developers
- pyMDOC-CBOR - package was started to give guidance to the Italian
community of developers implementing the wallet solution. The community has
decided not to implement MDOC-CBOR for now, but it is a requirement so
development will continue.
- There was a new release of pyXMLSecurity; waiting for new release for
pyff - try to discuss these next time
2 - AOB
- Next meeting 26 September. Giuseppe not sure if he will be there.
Thanks!
Shayna Atkinson
