*Idpy meeting 23 January 2024*
Attendees: Johan W, Shayna, Ivan, Matthew E.
0 - Agenda bash
1 - Project review
a. General -
We would like to get Roland to do a demo on his wallet system. There is
great interest about the trust within the system, the issuers, the
credentials, etc. Shayna will message him about scheduling it.
b. OIDC libraries - https://github.com/IdentityPython (idpy-oidc,
JWTConnect-Python-CryptoJWT, etc)
- Focus is currently on projects around oidc that implement the wallets.
- version 3.0.0 of idpy-oidc was released 17 December.
- side news - Christos is creating a repo to host GÉANT Core AAI
frontend, which is based on idpy-oidc. The main component is
SATOSA - this
will happen soon.
c. Satosa - https://github.com/IdentityPython/SATOSA
- Repo needs to be cleaned up. Some things have been merged but no
release. After cleanup, Ivan needs to add dependencies for oidc
backend as
an extra requirement (optional dependency).
- Other PRs that are pending to go in:
- Roland's work on removing SAML parts -
https://github.com/IdentityPython/SATOSA/pull/442 -
- Pavel's PR - https://github.com/IdentityPython/SATOSA/pull/419
- Johan's PRs - https://github.com/IdentityPython/SATOSA/pull/449
and https://github.com/IdentityPython/SATOSA/pull/450
- Ivan is looking at these Metadata PRs -
https://github.com/IdentityPython/SATOSA/pull/438 and
https://github.com/IdentityPython/SATOSA/pull/448 - Ivan has some
issues with these - there is a better way, using this (metaform) instead
as a base : https://github.com/SUNET/swamid-satosa/tree/main/src
- 438 is getting info from oidc services to be accessed by user -
denotes different metadata in terms of language in a specific
way, which is
also something that the oidc standard specifies. It's a nice
notation that
gets us away from having to nest things based on a language signifier.
Instead, this uses display name of a service with a language
tag - default
(no language tag) is english. Only does this for some
information, but more
would be useful and there will be some work to accommodate that.
- 448 - work is from Vladimir in NZ- mixing attibutes for person
with the metadata, because there is no other place for this
information. A
new slot should be dedicated for this information. Also there is some
information such as display name, scopes, etc., that is
implied to be about
the IdP, but it could be about the SP - there is no
indication where it
came from. The idea is to have a dedicated slot - not mix
things with the
attributes of the subject, and then under its own key also
have info for
the service AND for IdP. Then everything is separated and
made clear. And
then we can denote the languages with the notation from 438
(inherited from
oidc spec). Can be based on metainfo (see above) which has
collators that
work with both SAML and OIDC. For OIDC it is implied there is a data
structure - someone who has statically registered the client
would need to
add this data - it is not part of the official spec.
- Discussion of using Scott K's attribute store plugin response
microservice for IdP metadata attributes. It pulls from
metadata store in
the context. Be careful if you redirect - the object with the
information
will go away. Part of the reason Ivan would like proper
server side storage
for the proxy is that some federations may have many scopes
for their IdP
which creates a huge payload, and if you grab the metadata
from this object
and try to put it in a cookie, it is too large.
- Ivan would like to make this part of SATOSA, not just a
microservice, so not just attributes of user are available,
but also of the
service and the issuer. This will require proper APIs for
both the backends
and frontends, to get the data in a specific, defined format.
d. pySAML2 - https://github.com/IdentityPython/pysaml2
- Some updates need to be done for dependencies - there will be a new
release to accommodate these things
- cryptography has a new version
- cryptojwt
- xmlschema - new version/release is breaking APIs that are
currently being used.
- Fixing datetime() calls for python 3.12 as they are deprecated
- Get back to configurable encrpytion algorithms / signing
e. Any other project (pyFF, djangosaml2, pyMDOC-CBOR, etc)
- djangosaml2 - possibly looking to add new logging calls to do
tracebacks?
- thiss - working on container to get readymade setup
- Matthew E. is also trying to get an all-inclusive seamless access
deployment working, and once they do they will contribute their
containers/changes back to the community.
- His goal is to create a docker compose file that others can use,
using as a default eduGain metadata, to get their own seamless access
working. Then put pyff, thiss-mdq and thiss-js in the docker official
images library to get regular rebuilds for security updates
and follow good
current practices for containers.
- Johan may be able to share something with Matthew from his own
deployment, for smaller federations. They restart pyff
service every 15
minutes to reload metadata.
- pyff can be used two different ways: as a service, or as a
process that runs once and processes metadata, spitting out
xml files for
you, along with a json feed which can be used by
thiss-mdq/thiss-js. There
may still be a memory leak in the service variant?
- SWAMID is wanting to get a new release of pyff - there are some
small commits to HEAD that they need.
2 - AOB
Attendees: Johan Wassberg, Shayna, Ivan, Roland, Matthew E., Hannah
0 - Agenda bash
1 - Project review
a. General - Welcome to Johan Wassberg - working with SWAMID
b. OIDC libraries - https://github.com/IdentityPython (idpy-oidc,
JWTConnect-Python-CryptoJWT, etc)
- Roland has been working on a front end to SATOSA to be a credential
issuer for a digital wallet system - it is built and running!
- It has a number of endpoints - an authentication endpoint which can be
mapped to a number of backends, but also a credential
endpoint - to receive
a credential. Initially this used a SATOSA backend connected
to credential
constructor. This could be considered "twisting the arm" of
SATOSA , so
instead there is now an API with a built-in credential
constructor. Using
the API you could also call a service outside SATOSA for an
authenticated
user with an id token. The credential issuer uses a trust
pattern , which
is the OpenID federation. Roland says he should probably make
a OP frontend
without the credential issuer stuff in it , but with the
federation stuff
in it. If you want to test, you need a number of entities : SATOSA, a
wallet, and a federation with at least a trust anchor, wallet
provider and
a PID issuer (or some other credential issuer).
- Roland is writing documentation from the digital wallet point of view.
He wants to write it so that different audiences have their
own targeted
documentation; for example one section for IT architects or
CIOs, another
for deployers, another for developers who wish to extend the
capabilities.
- Discussion on how to register an OIDC RP with satosa-oidcop
- using *https://openidconnect.net/* <https://openidconnect.net/> (test
RP) and entering
*https://federation.scienceforum.sc/.well-known/openid-configuration*
<https://federation.scienceforum.sc/.well-known/openid-configuration>
for configuration, Matthew E receives "response type is not
registered"
errors.
- client needs to define property which indicates what response types
are allowed - by default it seems nothing is set. Matthew E
also reported
looking in database and saw no clients. May need to
statically register a
client - add an entry with Client id, client secret, client uri, and
probably some structure that defines response types that
client should be
using. May need to get Giuseppe involved. There doesn't seem to be an
example with the repo, and Matthew has looked through the
code and can't
seem to get it working correctly.
- Roland says there are a number of classes you can use to implement the
client database - it behaves like a dictionary. Roland could possibly
provide a configuration file that provides hints how to implement. For
instance, with the credential issuer frontend he didn't want to use
mongoDb, so he used a simple persistent layer with a
key-value database
behind it - implemented where the file name is the attribute
name, values
are json within the file. He can provide an example of this.
It would be
easy to add a client in this case - just drop a new file in a specific
directory, with a name that follows a convention and the
content as json
with the appropriate attributes (client id, client secret,
redirect uri)
c. Satosa - https://github.com/IdentityPython/SATOSA
- Many things are pending/need review and Ivan will look at them soon.
- New PRs from Johan:
- https://github.com/IdentityPython/SATOSA/pull/449 - current
implementation only generates parts of the metadata - doesn't
include all
backends/frontends
- https://github.com/IdentityPython/SATOSA/pull/450 - want to be able to
set a preferred backend when generating metadata where
multiple backends
are available - problematic because routing is based on which
endpoints the
request comes through. In other words: One of the proxies SUNET is
operating is set up such that there are multiple back ends
and these are
combined into one front end. There are multiple entity
descriptors with no
root. Also, the endpoints within the descriptors contain all
the backends.
It would be better to be able to select which endpoints of
which back end
are included in the metadata of the front end - a new option for a
preferred backend. Changes current behavior of the proxy, but
this problem
needs to be addressed. Ivan needs to experiment and recreate
this SUNET
environment. This opens up a new concept for SATOSA -
multi-tenancy. You
would be able to specify which back ends can talk to which
front ends -
multiple groups - where policies are applied. This is a
complex concept to
implement.
- https://github.com/IdentityPython/SATOSA/pull/451 - extend SATOSA
support paths beyond the root of the domain - Kristof created a new PR
after closing old one, with a clearer path on how to move forward
- https://github.com/IdentityPython/SATOSA/pull/438 - enriching data
structure of SATOSA with metadata information about oidc services. Adds
some things using conventions from oidc, but it is missing some useful
information that should be there - it's not taking into account how those
things are named/treated in the SAML world. You need to know
first whether
this is a SAML service or OIDC service to determine how to represent
things. Ivan will write a review. Hoping to get Pavel to do some of the
work.
- https://github.com/IdentityPython/SATOSA/pull/419 - involves
IsPassive/ForceAuthn in SAML, prompt in OIDC. Prompt is usually mandatory
when offline access is requested. Pavel wants to proxy this
concept through
ForceAuthn and IsPassive - if set, then prompt is skipped; or
reverse - if
prompt is no, IsPassive is set in SAML request to IdP. And when
ForceAuthn
is there - Internally store the association , then set with
ForceAuthn. The
work to make it configurable is done. Needs review to make sure it done
correctly. This is important for legal reasons - prompt is used
as consent.
Want to make sure that the new capability is available, but that the
defaults don't interfere with what is already in existence.
- https://github.com/IdentityPython/SATOSA/pull/452 - moving away from
SHA1 , removed from xmlsec. Needs to be removed from places it is used in
SATOSA as default. The PR allows new parameters to specify which
algorithms
to use. Needs some review and guidance, but should go in soon. Ivan also
needs to see what xmlsec is doing.
d. pySAML2 - https://github.com/IdentityPython/pysaml2
- Not much activity, but there are two issues that are annoying.
- https://github.com/IdentityPython/pysaml2/issues/944 - problems with
Windows temporary files . There is a template that acts as a
configuration
as to how a request or response should be signed. A temporary file is
written to (the temp file is an actual class within Python
ecosystem / core
Python libraries). Then xmlsec reads from this file to know
what to do.
Once a temporary file (that was created through the core
Python library) is
opened, Windows won't let other processes open the file and
read it. Python
should be addressing this rather than the project, but it's
not happening.
Ivan has a workaround that was never pushed - taking
advantage of Python
garbagecollector. Ivan will push and have the user test.
- https://github.com/IdentityPython/pysaml2/issues/946 - race condition.
Pysaml2 keeps state internally - in either an in-memory
dictionary or a
shelve file (Python-related) - a binary file that can be shared across
processes. User is using the shelve module for state in a
multi-processing
service. Processes are trying to access the file at the same time. One
process errors out and then the shelve module is corrupted.
Solution would
be to expose the state and manage it through a database or
some other way.
An interface is needed for this state, with proper adapters
to be able to
write or read from the storage back ends. Then using the
shelve module or
the in-memory dictionary (mySQL, Rides. etc) is a
configurable option. This
PR needs work to build that interface, carefully. Ivan will write a
response to this, but there are other priorities right now.
- There are some other pending PRs.
- There are some commits on the master branch and they will be creating
a new release.
e. Any other project (pyFF, djangosaml2, pyMDOC-CBOR, etc)
- Ivan knows there was discussion on djangosaml2 - adding support for
the content security policy headers. Not sure if there was a new release.
- pyff - there was a patch release previously.
- no activity on pyMDOC-CBOR - not clear if CBOR will be used as one of
the formats.
2 - AOB
- Matthew E has been using github actions -for a personal project, he
has a working github actions workflow that does linting and formatting
checks (purely style and syntax), testing (using pytest), a semantic
release build, and PyPI /github packages publishing.
- not sure how to do branch protection rules - Matthew will get this
figured out
- He will come back with some proposed configs ( aiming for 2 weeks)
- can use github oidc connector to authorize PyPI to get credentials
- doesn't need PyPI login. Ivan would just need to authorize the git
repositories, github action workflows and environment within the repo
- idpy not using travis CI anymore - it wasn't working
- Matthew is willing to offer his work in this area - can set up
something using TestPyPI. These are his pre-commit and ci yml files.
- https://github.com/irtnog/lethbridge/blob/main/.pre-commit-config.yaml
-
https://github.com/irtnog/lethbridge/blob/main/.github/workflows/ci.yml
- pysaml2 - Ivan says there are 2 PRs to introduce using githubactions
and pre-commit configuration. He has not had time to look into them. To run
tests on pysaml2, run "poetry pytest" and they also use tox.
- https://github.com/IdentityPython/pysaml2/pull/816
- https://github.com/IdentityPython/pysaml2/pull/882
- SATOSA - Ivan thinks we should do the same here. Decide on what the
configuration should look like. SATOSA also uses pytest with the options
you want. Has not been converted to use poetry. SATOSA would need more work
- linting etc has not been applied.
- pysaml2 had all this stuff incorporated in one big commit.
Sorry, I left the wrong date at the top of the agenda for the idpy
developers meeting. It should say "Tuesday, 9 January 2024, 14:00 UTC"
Thank you,
Shayna Atkinson
SCG Consulting Group
Dear all,
I’m pleased to announce a long overdue update to the Docker Official
Image for SATOSA, which includes several significant changes:
- SATOSA 8.4
- Python 3.12
- Debian 12 “bookworm”
- Alpine Linux 3.19
The updated container image is now available from Docker Hub. For more
information, see https://hub.docker.com/_/satosa.
Happy holidays!
Matthew
--
"The reason that ed is the standard editor is to remind you that
things could be worse, and once were." -- Tim Lavoie in comp.lang.lisp
*Idpy meeting 19 December 2023*
Attendees: Shayna, Ivan, Roland, Matthew E.
0 - Agenda bash
1 - Project review
a. General -
b. OIDC libraries - https://github.com/IdentityPython (idpy-oidc,
JWTConnect-Python-CryptoJWT, etc)
- idpyoidc 3.0.0 released
- openid federation implementation - Roland has openid4v (wallets) -
assumed openidconnect at base but it is oauth.
- Reminder that client registration information is not
well-documented.
- SATOSA container generates xml so that a newbie can get it going;
intent is to have a docker compose project that can stand up an oidc
environment and showcases all different integrations
- Roland has SATOSA OP frontend that is sort of based on Giusppe's
work, but partly new. He has the federation support. Didn't want to be
dependent on MongoDb, has persistence layers with store/fetch. Ideally
there is a function to register an OP (independent of backend).
- Ivan acknowledges that registration process is not well
documented and assumes pre-existing knowledge. Also does not have good
examples. He has not really used the dynamic registration,
but has used
static registration with the database for client data. He has used
different authentication methods for the clients: basic auth or client
secret post. Beyond that things get strange. Basic use cases
are there and
some advanced ones, but some may be missing. The problem is
there is not
just one level of documentation - are you a user, a
developer? Roland has
worked hard on base documentation for idpyoidc at
https://idpy-oidc.readthedocs.io/ but it needs to be updated.
This is for the library. The front end is one layer above and
we could use
documentation for that as well.
- openid federation adds a whole different layer to this. Clients
no longer register, they just become members of the federation.
c. Satosa - https://github.com/IdentityPython/SATOSA
- Considering how to expose metadata in openid federation format
- Some work required before making a new release
- Need to pull in Roland's PR separating pyoidc and pysaml2 from core
- Looking at PR on plugin for getting session info from SAML
entities - but the project tries to keep plugins protocol
agnostic (this is
clearly SAML specific).
- This brings up: how do you map from oidc to SAML? Right now
claims and attributes are mapped; scopes are there but not
really reflected
in internal data. Acr claim and amz claim are also not
reflected in
internal data. Need a document on how to map between the
protocols and
derive the internal data. This is needed if we want to
extend the core with
more protocols.
- jwt token from saml transaction (there is a spec)
d. pySAML2 - https://github.com/IdentityPython/pysaml2
e. Any other project (pyFF, djangosaml2, pyMDOC-CBOR, etc)
- Matthew working on getting SeamlessAccess pieces into one
release-ready container. If you have feet in both oidc and SAML worlds,
how do you take advantage of scalable federations in both with a
consistent
user experience? He's been coming up with ideas mostly surrounding
mirroring in SATOSA, to expose oidc as SAML. It is tricky, treating the
oidc entities the same as saml identities.
- Discussion of how to handle this; pictures attached.
2 - AOB
- Next meeting 9 January and then every 2 weeks after that
