feat(server): keep monitors and in-flight runs going when a licence lapses
Monitor execution and the workflow runner are deliberately unguarded: billing state must not take away a customer's ability to know their infrastructure is on fire, and killing a run midway leaves a half-configured server. The plan called for a server-limit check in gRPC Register. Left out: a server row only comes from CreateServer, which already checks the cap, so counting in Register counts the caller itself and would reject a legitimate agent at exactly the cap.
This commit is contained in:
@@ -26,6 +26,16 @@ type vantageServer struct {
|
||||
pb.UnimplementedVantageServer
|
||||
}
|
||||
|
||||
// Register carries no licence check, deliberately.
|
||||
//
|
||||
// A server row only ever comes from CreateServer, which checks the cap before
|
||||
// issuing a pre-registration token. By the time an agent calls Register its row
|
||||
// already exists, so counting here would count the caller itself: an instance
|
||||
// sitting exactly at its cap would reject the very agent it just authorised, and
|
||||
// every re-registration after a reinstall would fail too.
|
||||
//
|
||||
// The cap is enforced where rows are created, which is the only place it can be
|
||||
// enforced correctly.
|
||||
func (s *vantageServer) Register(ctx context.Context, req *pb.RegisterRequest) (*pb.RegisterResponse, error) {
|
||||
agentToken, err := services.RegisterServer(req.ServerId, req.PreRegToken, req.Hostname, req.IpAddress, req.OsInfo)
|
||||
if err != nil {
|
||||
|
||||
@@ -27,6 +27,11 @@ func loop(ctx context.Context) {
|
||||
active := map[string]*runner{}
|
||||
var mu sync.Mutex
|
||||
|
||||
// Monitors run regardless of licence state, deliberately.
|
||||
//
|
||||
// A customer whose card failed must not lose the ability to know their
|
||||
// infrastructure is on fire. Creating and editing monitors is blocked by the
|
||||
// API gate; executing the ones that already exist is not.
|
||||
sync := func() {
|
||||
monitors, err := services.ListServerScheduledMonitors()
|
||||
if err != nil {
|
||||
|
||||
@@ -202,6 +202,9 @@ func runServer(instanceID, runID string, srvIdx int, steps []models.ResolvedStep
|
||||
allSecrets := map[string]string{}
|
||||
serverFailed := false
|
||||
|
||||
// A run already in flight when the licence expires finishes its remaining
|
||||
// steps. New runs are blocked at the API, but killing a workflow midway leaves
|
||||
// a server in a half-configured state, which is worse than letting it complete.
|
||||
for i, step := range steps {
|
||||
startStep(runID, serverID, i, "running")
|
||||
stepStart := time.Now()
|
||||
|
||||
Reference in New Issue
Block a user