[pve-devel] [PATCHSET v2] API Tokens

[Fabian Grünbichler]

this is v2 of a patch set which aims to introduce API tokens into PVE.

the basic idea is to allow users to generate API token values that
- are attributed to this users
- easily revokable
- possibly less privileged than the user itself
- allow direct API calls without round-trips to create/refresh a ticket

token information is stored in user.cfg together with the other
access-control information. the actual token values are stored in a
'shadow' token.cfg file under /etc/pve/priv, with verification happening
over a special IPCC call.

high-level changelog v1->v2:
- incorporated review
- added shadow token.cfg + verification via IPCC
- API refinement
- pveum integration
- GUI integration 0.1
- new permissions API call
- fixed test cases

still missing:
- thorough review ;)
- PMG adaption (at least for the changed method signatures in
  pve-http-server)
- checking API endpoints for 'notoken'-ification

I tried to order independent clean-ups etc. up front with-in each repo,
as usual.

p.s. don't judge me too hard for my lack of JS foo / blatant copying of
existing code ;) also, I am not very happy with the 'icon' used for API
token in the GUI, if someone knows a better one I am all ears :)

pve-access-control:

Fabian Grünbichler (23):
  user.cfg: ensure propagate flag is 1/0 when parsing
  user.cfg: sort group and pool members, role privs
  pveum: add list commands
  access-control: remove check_permissions/permission
  rpcenv: drop unused roles()
  auth: pull username REs into variables
  refactor acl transformation code
  API token: add REs, helpers, parsing + writing
  API token: add check_token_exist API helper
  API token: add (shadow) TokenConfig
  API token: add verification method
  API: add API token API endpoints
  API: add group and token info to user index
  API: include API tokens in ACL API endpoints
  API token: implement permission checks
  api: mark some paths notoken
  roles()/permissions(): also return propagate flag
  API: add 'permissions' API endpoint
  API token: add tests
  tests: unify config file naming
  API: add group members to group index
  pveum: add 'pveum user token add/update/remove/list'
  pveum: add permissions sub-commands

 PVE/Makefile                     |   1 +
 test/Makefile                    |   1 +
 PVE/API2/ACL.pm                  |  30 ++-
 PVE/API2/AccessControl.pm        |  60 ++++++
 PVE/API2/Group.pm                |   7 +
 PVE/API2/User.pm                 | 315 ++++++++++++++++++++++++++++++-
 PVE/AccessControl.pm             | 311 +++++++++++++++++++++---------
 PVE/Auth/Plugin.pm               |   5 +-
 PVE/CLI/pveum.pm                 |  83 ++++++++
 PVE/RPCEnvironment.pm            | 120 ++++++++----
 PVE/TokenConfig.pm               |  79 ++++++++
 debian/control                   |   2 +
 test/dump-perm.pl                |  16 +-
 test/perm-test1.pl               |   9 +-
 test/perm-test2.pl               |   4 +-
 test/perm-test3.pl               |   4 +-
 test/perm-test4.pl               |   4 +-
 test/perm-test5.pl               |   4 +-
 test/perm-test6.pl               |  56 ++++--
 test/perm-test7.pl               |  26 ++-
 test/perm-test8.pl               |  68 +++++++
 test/{user.cfg.ex1 => test1.cfg} |   0
 test/test8.cfg                   |  28 +++
 23 files changed, 1043 insertions(+), 190 deletions(-)
 create mode 100644 PVE/TokenConfig.pm
 create mode 100644 test/perm-test8.pl
 rename test/{user.cfg.ex1 => test1.cfg} (100%)
 create mode 100644 test/test8.cfg

base-commit: 4ef92d0dabc95e302b119d1120449945982962d2


pve-cluster:

Fabian Grünbichler (2):
  pmxcfs: add verify_token IPCC request
  cluster: add priv/token.cfg to observed files

 data/src/cfs-ipc-ops.h |  2 ++
 data/src/server.c      | 58 ++++++++++++++++++++++++++++++++++++++++++
 data/src/status.c      |  1 +
 data/PVE/Cluster.pm    | 19 ++++++++++++++
 4 files changed, 80 insertions(+)

base-commit: 0e578bb7bbbf03b2d6c9d8e8c6c4d1db4c530567


pve-common:

Fabian Grünbichler (1):
  API schema: add 'notoken' property

 src/PVE/JSONSchema.pm | 5 +++++
 1 file changed, 5 insertions(+)

base-commit: 450d7efcd175c082815407199113c5d71c78c846


pve-http-server:

Fabian Grünbichler (1):
  api-server: extract, set and handle API token header

Tim Marx (1):
  allow ticket in auth header as fallback

 PVE/APIServer/AnyEvent.pm            | 34 +++++++++++++++++++++-------
 PVE/APIServer/Formatter.pm           | 21 ++++++++++++-----
 PVE/APIServer/Formatter/Bootstrap.pm |  1 +
 3 files changed, 42 insertions(+), 14 deletions(-)

base-commit: aa6e7a0d639d58172612b55d892535e098648c25


pve-manager:

Fabian Grünbichler (9):
  auth_handler: handle API tokens
  rest_handler: implement 'notoken' API endpoints
  pveproxy: use new cookie extraction method
  api/tasks: attribute token tasks to user
  www: add 'users' columns to Groups model
  www: add permissions button to userview
  www: add Token Panel + Edit Window
  www: add Token to ACL
  www: add TokenView with fixed userid

 www/manager6/Makefile              |   4 +
 PVE/API2/Cluster.pm                |   3 +
 PVE/API2/Tasks.pm                  |  15 ++
 PVE/HTTPServer.pm                  |  60 ++++---
 PVE/Service/pveproxy.pm            |   2 +-
 www/manager6/Workspace.js          |  10 ++
 www/manager6/dc/ACLView.js         |  23 ++-
 www/manager6/dc/Config.js          |   8 +
 www/manager6/dc/GroupView.js       |   6 +
 www/manager6/dc/PermissionView.js  | 167 ++++++++++++++++++
 www/manager6/dc/TokenEdit.js       | 125 +++++++++++++
 www/manager6/dc/TokenView.js       | 275 +++++++++++++++++++++++++++++
 www/manager6/dc/UserView.js        |  14 +-
 www/manager6/form/GroupSelector.js |   8 +-
 www/manager6/form/TokenSelector.js |  91 ++++++++++
 15 files changed, 781 insertions(+), 30 deletions(-)
 create mode 100644 www/manager6/dc/PermissionView.js
 create mode 100644 www/manager6/dc/TokenEdit.js
 create mode 100644 www/manager6/dc/TokenView.js
 create mode 100644 www/manager6/form/TokenSelector.js


base-commit: 1e18c15ad18ca528ce353ed270eda0b464ea098f